What Context Engineering Actually Is
Prompt engineering asks one question: how should I phrase this? Context engineering asks a different one: what does the model actually need to know to do this well?
The term itself has a real, traceable origin. Shopify's CEO Tobi Lütke described it in June 2025 as "the art of providing all the context for the task to be plausibly solvable by the LLM." Days later, AI researcher Andrej Karpathy popularized it further, calling it "the delicate art and science of filling the context window with just the right information for the next step." Anthropic later gave it a clean, technical definition of its own: prompt engineering is about writing and organizing instructions, while context engineering is the set of strategies for curating and maintaining the optimal set of tokens during inference, everything the model sees, not just what it's told to do.
In plain terms
A prompt is one instruction. Context is everything surrounding that instruction, memory from past conversations, documents retrieved for the task, tool definitions, conversation history, and the rules the system runs on before it ever sees your actual question.
| Prompt Engineering | Context Engineering | |
|---|---|---|
| Main question | How should I phrase this? | What does the model need to know? |
| What it manages | The instruction itself | Memory, retrieved data, tools, history |
| Where it matters most | A single, one-off question | Long, multi-step agent workflows |
| Typical failure mode | A poorly worded prompt | A model missing the right data at the right step |
| Still needed today? | Yes, but a smaller part of the job | Yes, and it's now the larger part |
Why the Shift Actually Happened
For a while, a clever prompt really did move the needle. Phrases like "think step by step" measurably improved GPT-4's output back in 2023. That effect has largely worn off. Modern models were trained on millions of examples of exactly those tricks, so the tricks stopped being special, they became the baseline.
What didn't stop mattering is what the model actually has in front of it. Multiple people building production AI systems now describe the same rough split: the instruction itself is maybe 5% of what determines a good answer from a real, deployed AI agent. The other 95% is context, the memory, the retrieved documents, the tool definitions, and the conversation history the model is working with at that exact step.
A few widely-used prompting tricks have specifically stopped paying off
- Role-play preambles ("You are an expert...") barely move accuracy on frontier models anymore, they mostly just spend tokens
- Incantations and bribes ("I'll tip you," "or you'll be penalized") have no measured, durable benefit, modern models follow plain instructions just as well without the theatrics
- Megaprompts, stuffing in every instruction "just in case," don't help either, more tokens isn't more accuracy, and can actively hurt it
One 2026 industry survey, the State of Context Management Report, found that 82% of IT and data leaders now agree prompt engineering alone isn't enough to run AI at real scale, and 95% of data teams planned to invest in context engineering training during 2026. Numbers like that vary by survey, but the direction they all point in is consistent.
The Building Blocks of Context Engineering
Context engineering isn't one technique, it's managing several layers of information at once:
- System instructions - the foundational layer: who the model is, what rules it follows, what format it should use. This used to be the entire job of prompt engineering. Now it's just the starting point
- Memory - what the system remembers from earlier in this session, or across past sessions entirely
- Retrieved context - documents, records, or data pulled in specifically for this task, often through a retrieval system (commonly called RAG, retrieval-augmented generation) rather than stuffed into the prompt permanently
- Tool definitions - what tools or functions the model can actually call, and what each one does
- Conversation history - everything said so far in the current session that's still relevant
At step one of a task, prompt engineering and context engineering look similar. By step 47 of a long agent workflow, they don't. What matters that far in isn't how the original instruction was phrased, it's what the agent still remembers, what it retrieved along the way, what tools it has left to use, and how much context window space remains. Prompt engineering has no real answer for step 47. Context engineering is built specifically to.
The Real Trap: Context Rot
More context is not automatically better context. Researchers have documented something often called the "lost-in-the-middle" problem: models handle information placed at the very beginning or the very end of their context window more reliably than information buried in the middle. Pile on too much, and a model's ability to recall any one specific piece actually drops, sometimes called "context rot."
That means the real skill isn't maximizing how much you feed the model. It's curating it, deciding deliberately what belongs in the context window for this specific step, and what gets deliberately left out.
How to Actually Do This
- Curate, don't dump. Feed the model what this specific step needs, not everything you have available
- Structure clearly. Well-organized system instructions and clearly labeled sections still matter, they're the foundation context engineering builds on, not something it replaces
- Manage memory on purpose. Decide what should persist across a session or across sessions, and what should reset, rather than letting everything accumulate by default
- Match retrieval to the task. Pull in the specific documents or data a task needs at that moment, instead of keeping a static, growing pile of "just in case" information in every prompt
- Watch your context budget like a real resource. Track how much of the window is being used by memory, retrieved data, and tool definitions, since none of that is free, and all of it can crowd out what actually matters for the next step
The Bottom Line
Prompt engineering isn't dead, exactly, clear instructions still matter, and always will. What's changed is that clear instructions are no longer the hard part. The hard part, and the actual skill worth learning in 2026, is deciding what the model sees at every step of a real task: what it remembers, what it retrieves, what tools it has, and what gets deliberately left out. That's context engineering, and it's already the difference between an AI demo that looks impressive once and a system that actually works reliably in production.
Based on the original public framing of the term by Tobi Lütke and Andrej Karpathy in 2025, Anthropic's own technical definition of context engineering, and multiple 2026 industry reports and practitioner writeups tracking the shift. This is a fast-evolving area of AI engineering, terminology and best practices are still settling.
