In-Context Learning

Programs like ChatGPT receive their instructions as text that you type in. In this text, you can write not only a question but also a few solved examples. So you first show two or three cases with the desired answer and then pose the actual task. The program recognizes the pattern from this and handles the new task in the same way. This exact behavior is called in-context learning, meaning learning from the supplied text. Importantly, the program itself does not change in the process. As soon as the input is gone, what was learned is gone too.

Why examples in the text are cheaper than retraining

In the past, a model had to be specifically retrained for every new task. Retraining means changing the model’s internal numerical values using thousands of examples. This costs computing time, money, and experts. With in-context learning, a well-written text with three examples is often enough. This turns a research project into a ten-minute task.

This is why companies without an AI department were able to start using language models at all. A tax office can show the model how to sort invoices into a table. An editorial team can demonstrate headlines in their own style. There is no need for a specialized model, just a good input.

The second reason is scientifically interesting. With older programs, it was clear: what wasn’t trained couldn’t be done by the program. Large language models, by contrast, solve tasks that were never part of the training as a dedicated exercise. This ability only appeared once the models became very large. Such leaps are called emergent abilities in research.

Recognizing patterns instead of storing knowledge

At its core, a language model always does the same thing: it predicts which word is likely to come next. To do this, it looks at the entire text currently in front of it. If there are three examples following the pattern “input – answer,” the most likely continuation is a fourth answer in the same format. The model thus imitates the structure it sees in the text.

You can compare it to a musician who has practiced scales and pieces for years. If you play two bars for them, they can continue playing in the same style. They don’t learn anything new in the process, but rather use practiced skills for a given pattern. If they forget the bars, the effect is over.

This leads to the limits. The context, meaning the text window for input and answer, is limited. If only a few pages fit, you cannot send an entire textbook as a collection of examples. Furthermore, the order of the examples matters: sometimes just swapping them changes the answer. And genuine expert knowledge does not arise this way. Anyone who wants to teach the model facts it doesn’t know needs retraining or a connected database.

From the chat window to enterprise software

Anyone who writes to a chatbot “Phrase it like in this example” is using in-context learning. In technical texts, you’ll also come across the terms zero-shot, one-shot, and few-shot. Zero-shot means: only the instruction, no example. Few-shot means: a few examples are provided. These words frequently appear in model announcements and benchmark tables.

In products, this method is often built in invisibly. A support tool automatically sends along several sample answers before every customer question. The user sees none of this; the program assembles the input in the background. The profession that designs and tests such inputs is called prompt engineering.

A common misconception is that the model becomes permanently smarter through the examples. This is false. In the next chat without these examples, it behaves as before again. This also plays a role in market news: large context windows are a selling point for providers, because more examples and documents fit in.