Weights (Model Weights)

An AI program that writes texts or recognizes images consists, at its core, of a huge calculation rule. Very many numbers go into this calculation, and it is precisely these numbers that are called weights. They determine how strongly each individual piece of intermediate information influences the result. At the start they are set randomly, and the program then produces only nonsense. During learning, they are slightly readjusted millions of times until the outputs fit. You can picture them like the knobs on a mixing console: the setup of the console stays the same, but the position of all the knobs decides how it sounds.

Why the number file is the actual capital

The program code of an AI model is usually only a few thousand lines long and technically nothing special. What is valuable is the result of the learning process, i.e. the adjusted weights. For large language models, this learning costs many millions of euros in computing time and electricity. That is why companies talk about their weights the way others talk about a trade secret.

This is also connected to the much-discussed question of open models. If a company releases the weight files for download, anyone can run the model on their own hardware. Meta has done this with its Llama series, as have the French company Mistral and the Chinese company DeepSeek. OpenAI and Google, by contrast, withhold the weights of their strongest models and offer access only over the internet.

A common misconception: freely available weights do not mean that you can retrace the learning process. The training data and many details often remain secret. You get the finished result, not the recipe. Experts therefore more precisely call such models “open weights” rather than open source.

How random numbers become usable values

During learning, the model receives an example and makes a prediction. This is compared with the correct answer, and the error is calculated. The procedure then calculates backwards which weights contributed to the error. Each of these is shifted by a tiny amount in the better direction. This cycle runs billions of times until the predictions are reliably correct.

The number of weights is called the parameter count, and it is often built into the model name. A model with the suffix “7B” has around seven billion such numbers. Each number needs storage space, usually two bytes. Seven billion weights therefore take up about 14 gigabytes, which already nearly fills the graphics memory of an ordinary gaming PC.

Because this is expensive, the numbers are often stored more coarsely, for example with one byte or less per weight. This procedure is called quantization. As a result, the model becomes smaller and faster and loses only a little accuracy. Conversely, you can take someone else’s weights as a starting point and further adjust them with your own data. This retraining is called fine-tuning, and it is much cheaper than training from scratch.

Weights in the news and on your own computer

In news about AI, weights mostly come up when it is about access and control. Headlines like “model releases the weights” mean: the model can be used further without the manufacturer’s permission. The term is also central in security debates, because theft of the weight files would amount to theft of the entire product. EU regulation and US export rules are, among other things, precisely concerned with which weights may go where.

In practice, you encounter them on platforms like Hugging Face, where thousands of models are available for download. There, the file list often shows a format called safetensors, and that is nothing other than a collection of weights. Anyone who runs such a model on their own laptop loads exactly these numbers into working memory.

It is important to distinguish this from the prompt, i.e. the user’s input. The prompt does not change the weights. A chatbot does not permanently learn from your conversation as long as the manufacturer does not initiate new training. Everything the model can permanently do is fixed in the weights.