Context Engineering

Programs like ChatGPT respond to text that is sent to them. This text is their only view of the world: whatever isn't in it, they don't know at that moment. The entire amount of text that such a system has in front of it for a given request is called the context. Context Engineering refers to the work of deliberately assembling this context. This includes the actual question, examples, excerpts from documents, previous messages, and rules for behavior. In other words, you build the information folder that the system looks at before it starts writing.

Why the folder matters more than the model

In practice, AI applications rarely fail because the language model is too dumb. They fail because it was missing a crucial piece of information. An assistant that doesn't know a company's vacation rules will make them up. Experts call such fabricated statements hallucinations. They often disappear abruptly once the right documents are placed in the context.

The context is also limited. Every system has a maximum amount of text it can process at once, measured in tokens – word fragments about four characters long. Modern systems can handle several hundred thousand of these, i.e. entire books. Still, the space isn't free: every additional token costs computing time and thus money.

On top of that comes an effect researchers call "Lost in the Middle." Information at the beginning and end of a long context is used more reliably than information in the middle. More text therefore doesn't automatically mean better answers. Dumping in twenty documents instead of selecting the two right ones often makes the result worse.

What good context is made of

At the very front is usually a system instruction. It sets the role, tone, and boundaries, for example: "You are a customer service assistant and do not give legal advice." This is often followed by a few example answers so the format is clear. This technique is called few-shot prompting and often works better than a long description in words.

The largest part usually consists of retrieved documents. For this, a method called RAG is used, short for Retrieval Augmented Generation. A search function finds the passages relevant to the user's question from a database. Only these passages are moved into the context, not the entire database. This way, the model gets up-to-date company knowledge without needing to be retrained.

With long conversations, another problem arises: the history grows with every message. At some point, it no longer fits. Common solutions are summarizing older sections or a notes store from which only relevant points are retrieved. Context Engineering is therefore not a one-time act of writing a text, but an ongoing management of storage space.

From prompt tinkering to its own job profile

Around 2023, people mostly talked about Prompt Engineering, meaning the skillful phrasing of individual questions. The term Context Engineering has since largely replaced it. The reason: in serious applications, no one types by hand anymore. A program automatically reassembles the context for every request. Prompt Engineering is thus now just a subfield of the larger topic.

This becomes most visible with so-called agents. These are AI systems that carry out several steps in sequence, such as searching, calculating, and writing emails. After each step, a decision must be made about which intermediate results stay in the context. Providers like Anthropic and OpenAI now publish detailed guides on this.

Even as a regular user, you practice Context Engineering without calling it that. Someone who uploads an essay for correction and adds the grading criteria gets better feedback than with the mere request "Improve this." A common misconception is that polite phrases or dramatic wording help a lot. What really helps are concrete details: goal, target audience, length, and examples.