Foundation Model

In the past, a separate computer program was built for each task that learned from examples: one for translations, one for detecting spam emails, one for image captions. Each of these programs learned only its one task. A foundation model turns this principle on its head. It is first trained generally on enormous amounts of text, images, or program code from the internet, without a fixed goal. In doing so, it learns language, connections, and world knowledge. Only afterward is this one model deployed for many different purposes or tailored to them with a few additional examples. The name comes from the foundation of a house: you pour it once, and very different buildings can be erected on top of it.

One model for a thousand applications

The economic appeal lies in reuse. The base training is extremely expensive and can cost hundreds of millions of euros. However, these costs are incurred only once. After that, the same model can handle customer service, summarize contracts, or write software. Adapting it to a specific task often costs only a tiny fraction of that.

This is changing the power dynamics in the tech industry. Only a few corporations can afford the base training, because it requires huge data centers with specialized chips. OpenAI, Google, Meta, Anthropic, and a few Chinese companies dominate this field. Everyone else builds their products on top of other companies' foundation models. This is precisely why these names appear so frequently in stock market news.

There is also a downside. When thousands of applications rely on the same model, its errors and biases are passed on everywhere. If the model, for instance, distorts certain professions, this distortion reappears in every recruitment software built on top of it. Experts therefore speak of a concentration risk.

First learn generally, then tailor

The base training takes place without human labeling. For text, it usually works like this: the model is given the beginning of a sentence and has to guess the next word. If it’s wrong, its internal values are corrected slightly. These values are called parameters; large models have hundreds of billions of them. Repeating this billions of times produces, almost as a side effect, a remarkably broad understanding of language.

Fine-tuning follows next. Humans rate the model’s responses as helpful or useless, and the model learns to align itself with these ratings. Only this step turns a pure text continuer into a usable assistant. Companies can additionally retrain it on their own data, such as a bank’s internal handbook.

Often even less effort is needed. You simply give the model the task as an instruction in everyday language, possibly with two or three examples. The model itself remains unchanged in the process. This flexibility is the real difference from older AI systems, which had to be completely rebuilt for every new task.

Where foundation models work in the background

In everyday life, people usually encounter them without noticing. The text suggestions in an email program, the automatic summary of a video, the search answer given in full sentences: behind almost all of this is a foundation model from a major provider. Coding assistants like GitHub Copilot also draw on such models.

In news articles, the term frequently appears alongside the word language model. Both mean similar things but are not identical. A language model works only with text, while a foundation model can also process images, audio, or video. Every large language model is a foundation model, but the reverse is not true.

Politically, the term has also become entrenched. The European Union’s AI Act places special requirements on such models, because so many applications depend on them. Providers must document what they trained with and assess risks. Anyone who reads economic news will therefore regularly encounter this word.