Foundation Model

In the past, a separate computer program was built for each task, one that learned from examples: one for translations, one for recognizing cat images, one for spam filters. Each of these was narrowly tailored to its task and could do nothing else. A foundation model turns this approach on its head. It is first trained on enormous amounts of text, images, or program code in a very general way, without a fixed goal. In doing so, it learns very broad knowledge about how language and images are structured. Only afterward is it applied to individual tasks, and to many different ones at the same time. The English name means literally “foundation model”: it is the foundation on which other applications are built.

One foundation, many buildings on top

The economic appeal lies in reuse. The expensive base training happens once and costs, depending on size, tens or hundreds of millions. After that, the same model can run a customer service chat, summarize contracts, and complete program code. A company building an AI application therefore doesn’t have to start from zero. It rents access to a ready-made foundation model and adapts it using comparatively little of its own data.

This also explains why business news talks so much about just a few providers. Whoever owns such a model sits at a point through which a great many products pass. OpenAI, Google, Anthropic, Meta, and Alibaba are among the best-known developers. The data centers and graphics chips needed for training are one reason for the industry’s massive investments.

There is also a downside. When thousands of applications build on the same model, they all inherit its weaknesses. A bias in the training data or a typical tendency to make errors then doesn’t affect just one product, but many. Experts call this a concentration risk.

From raw training to a finished application

The first phase is called pretraining. The model is given enormous amounts of text and learns a seemingly trivial task: predicting the next word. Because it practices this billions of times, it must in the process learn grammar, factual knowledge, and relationships along with it. Nobody tells it what a verb or a capital city is. This structure emerges from the data.

Next comes fine-tuning. Here, the general model is refined using much smaller, carefully selected datasets. This is how it learns, for example, to respond helpfully instead of simply rambling on, or how it learns the specialized vocabulary of a doctor’s office. The effort required for this is a fraction of the pretraining. Often, plain instructions in the input field are enough, without changing anything in the model itself.

A common misconception is equating foundation model with language model. Language models are the best-known case, but not the only one. There are foundation models for images, for music, for protein structures, and for weather data. What matters is not the type of data, but the principle: train broadly, then apply versatilely.

Where these models are found in everyday life

Every chatbot you use in your browser runs on a foundation model. The same is true for autocomplete in programming tools, for automatic subtitles, and for image generators that turn a description into a picture. Even the search engine that shows you an answer instead of a list of links relies on it.

In news reports, these models usually appear under product names, such as GPT, Gemini, Claude, or Llama. When a “new model generation” is mentioned there, it almost always refers to a new foundation model. Legal texts such as the EU’s AI Act often use the term “general-purpose AI model” for this and attach special obligations to providers as a result.