Foundation Model

A foundation model is a very large computer program that has learned patterns from enormous amounts of text, images, or audio recordings. This learning from examples is called training, and in the case of a foundation model, it happens without a fixed intended purpose. The result is not a finished product but a kind of basic set of capabilities. On this basis, very different applications can be built: a chat program, a translation aid, an assistant for programmers. The English term for this is Foundation Model, and the image fits well. You pour a solid foundation once and later put various houses on top of it.

Why hardly anyone starts from scratch anymore

A few years ago, a separate small model was built for each task. Anyone who wanted to detect spam emails collected spam emails and used them to train a program. For the next task, the work began anew. Foundation models have turned this logic on its head: you train once, very elaborately, and then adapt the result many times over afterward.

The reason is primarily economic. Training a large foundation model costs tens or hundreds of millions of dollars, depending on its size, for computing time and electricity alone. Worldwide, only a few companies can afford this, such as OpenAI, Google, Meta, or Anthropic. Everyone else rents or downloads these models and builds their own products on top of them.

This creates a dependency that is much discussed. When thousands of companies build on a few models, their weaknesses affect everyone simultaneously. A flaw in the foundation propagates into every house built on top of it. This is also a political issue, because almost all of the most important foundation models to date come from the USA and China.

From raw training to ready-to-use application

The first step is called pretraining. The model is presented with vast amounts of text and must repeatedly guess which word comes next. Out of billions of such guessing attempts, knowledge of grammar, facts, and relationships emerges. No one teaches the model individual rules in the process; it derives them itself from the data.

This is followed by adaptation, known in technical jargon as fine-tuning. Here, the finished model is further trained with a comparatively small, carefully selected collection of data. A law firm can thus tune a general language model to legal texts. This second step costs only a tiny fraction of the pretraining, often just hours of computing time.

A common misconception: foundation model and chatbot are not the same thing. A freshly pretrained model does not hold a conversation; it merely continues text. Only additional rounds of training with human feedback turn it into a polite, helpful assistant. What you use in a chat app is therefore a foundation model plus several refinement steps.

Which foundation models you actually encounter

The best-known representatives bear names like GPT, Gemini, Claude, or Llama. If a news report says a company has released a new model, it is almost always about a foundation model. Image generators like Stable Diffusion also belong to this category, just for images instead of text.

In everyday life you usually don’t notice this, because the model works beneath the surface. The search summary on Google, the text suggestions in office software, the autocorrect on your phone: a foundation model is often behind these. Many companies don’t even disclose which one they use.

For investors and business journalists, the term is important because it explains where the money is flowing. Whoever owns a foundation model controls an infrastructure that others depend on. The European AI Act, the EU’s AI law, therefore treats particularly powerful foundation models as a separate category with stricter obligations.