Weights

At its core, an AI model is a very large computational rule. This rule contains millions or billions of individual numbers. These very numbers are called weights. They are not fixed from the start but are changed step by step during learning. In the end, everything the model can do is contained in these numbers. You can picture them like the setting of millions of tiny dials on a giant mixing console: the music does not come from the device alone, but from the exact setting of all the dials.

Why the numbers are the actual product

The program code of an AI model is usually manageable and often freely available. What is valuable is something else: the fully tuned weights. Producing them costs weeks of computing time on thousands of specialized chips and, in some cases, hundreds of millions of euros. Whoever has this file can run the model on their own computers without having to pay for that effort again.

That is exactly why the discussion about open AI revolves around this point. A company like Meta publishes the weights of its Llama models for download. Other providers such as OpenAI keep the weights of their largest models secret and only allow them to be used over the internet. The technical term for this is “open weights” as opposed to a closed model.

Published weights cannot be recalled. Once downloaded, anyone can redistribute them and also remove the built-in safety rules again. This is one of the main points of contention in the debate about AI safety.

How the dials find their setting

At the beginning, the weights are pure chance, and the model only produces nonsense. Then it is given an example, say half a sentence, and is supposed to predict the next word. The prediction is compared with the actual word, and the difference between the two yields an error value.

Now a procedure calculates how much each weight contributed to this error. Each weight is shifted a tiny bit in the direction that reduces the error. This process is called training and repeats many billions of times. The individual changes are so small that they go unnoticed. Only their sum turns noise into a model capable of language.

It is important to distinguish this from usage: once training is finished, the weights are fixed. A chatbot does not permanently learn anything during a conversation. It can only refer to what is in the text window during the ongoing conversation. New knowledge only enters the weights with a new training run.

Where weights show up in news and downloads

When reports mention a model with 70 billion parameters, they mean the weights. This number roughly indicates how large and how expensive the model is to run. But it does not reliably indicate how good it is: smaller, carefully trained models regularly outperform larger ones. A common misconception is that more weights are automatically better.

On platforms like Hugging Face, weights can be downloaded freely. The files are huge; a medium-sized model quickly takes up 15 gigabytes. To make it fit on a normal computer, the numbers are often stored more coarsely, for example with fewer digits per weight. This procedure is called quantization and saves memory, but costs a bit of accuracy.

Adjusting models is also about weights. In so-called fine-tuning, you take a finished model and only slightly alter its weights using your own data, for example to match a company’s language. And when an AI suddenly responds worse or differently, the provider has usually rolled out new weights. This explains why a chatbot can change overnight.