Language Model

A language model is a computer program that can continue texts. Beforehand, it has read enormous amounts of text: books, news, websites, forums. From this, it has learned which words typically follow one another. If you give it the beginning of a sentence, it calculates the most probable next word. Then it appends this word and calculates again from the start. This is how entire paragraphs come into being, word by word. The program doesn’t look anything up, then; it calculates with probabilities.

From autocomplete to an all-rounder

For a long time, this technique was considered a gimmick. The text suggestions on your phone are, at their core, the same principle, just much simpler. But starting around 2020, something surprising emerged: when such models are greatly enlarged and trained with far more text, capabilities suddenly appear that nobody built them for. They translate, summarize, write program code, and answer exam questions.

The reason lies in the task itself. To really predict the next word well, grammar alone isn’t enough. Anyone wanting to correctly complete the sentence “The capital of Japan is” must have learned world knowledge along the way. So the model stores a great deal of factual knowledge on the side, without anyone having taught it individually.

That is precisely why the term is relevant on the stock market today. Companies worth billions have emerged around language models, and demand for the matching computing chips has transformed entire industries. When the news talks about a new AI model, it almost always means a language model.

How text becomes a model

First, the text is broken down into small building blocks called tokens. A token is usually a short word or part of a word. Each token is assigned a list of numbers describing its meaning in relation to other words. Words with similar meanings lie close together in this numerical space.

Then training begins. The model is given a text excerpt in which the next token is hidden, and it guesses. Afterward, the correct answer is revealed. If the model is off, its internal adjustment knobs are tweaked slightly. These knobs are called parameters; large models have hundreds of billions of them. This process repeats billions of times over the course of weeks.

After this basic training comes a fine-tuning phase. Humans rate the model’s answers as helpful or unsuitable, and the model is adjusted toward the good answers. Only through this does a text machine become a usable assistant. A common misconception, by the way, is that the model looks things up on the internet. By default, it does not do this; it answers from whatever stuck in its parameters during training. That’s why it sometimes invents facts that sound convincing.

Language models in everyday life and in products

The best known are chat programs like ChatGPT, Gemini, or Claude. Behind each of these is a language model with a user interface. But the same technology is also found in less conspicuous tools: in translators, in the search function of search engines, in text suggestions in office programs, and in assistants that suggest code to programmers.

In companies, language models often run in the background. They sort customer inquiries, summarize long contracts, or extract figures from reports. Some companies operate smaller models on their own servers for this purpose, because their data is not allowed to leave the premises.

In news texts, you frequently encounter the English abbreviation LLM for Large Language Model. The same thing is meant. Pay attention to figures on the number of parameters or the context window, that is, the amount of text a model can take in at once. Both are common metrics that providers use to compare their models.