Parameter (AI model)

At its core, an AI model is a very large computational rule. Within this computational rule are numbers that are not fixed but can be adjusted. These adjustable numbers are called parameters. They determine how strongly an input value influences the result. At the start they are chosen randomly, so the model initially produces nonsense. During learning from millions of examples, they are gradually fine-tuned step by step. In the end, everything the model "knows" is contained in these numbers. Nothing else is stored: no text, no rules, no list of facts.

Why this matters

The parameter count is the industry's best-known metric. When the news talks about a "70-billion model," it means parameters. This number partly determines how much memory a model needs and how expensive it is to run. Roughly speaking: more parameters mean more capacity for patterns and relationships.

However, this rule of thumb has become unreliable. A smaller model with better training data often outperforms a larger one today. Moreover, not all parameters have to compute simultaneously. Some models activate only a fraction of them per request. The raw parameter count therefore says little about actual performance. Experts additionally look at benchmark results and the computational cost per response.

How parameters come into being

Imagine a mixing console with billions of dials. Each dial is a parameter. Training proceeds in rounds. The model is given a text excerpt and is supposed to predict the next word. The prediction is then compared with the actual word. From the discrepancy, a procedure calculates in which direction each individual dial should be moved. All dials are shifted by a tiny amount. Then the next round begins with the next text excerpt.

This process repeats billions of times over the course of weeks. No one sets a dial by hand. Nor can it be said which parameter is responsible for which piece of knowledge. The knowledge is distributed across many numbers simultaneously. This is precisely why it is so hard to understand why a model gives a particular answer.

Each parameter is stored as a number with decimal places. How precisely is a decision with consequences. Storing it more coarsely makes the model smaller and faster, but somewhat less accurate. This procedure is called quantization and makes large models usable on ordinary hardware.

Where the term appears

In product announcements, the parameter count often appears directly in the name. Model lineups carry suffixes like "7B" or "405B," where B stands for billion. Anyone wanting to try out a model on their own computer reckons with it directly: as a rule of thumb, a model with seven billion parameters requires several gigabytes of memory when stored more coarsely. That's why small models run on a laptop, while large ones only run in data centers.

The number also plays a role in financial news. Larger models require more graphics cards, more electricity, and more capital. Analysts therefore use parameter counts as an indicator of a company's investments. And when a provider releases a model with "open weights," it means the trained parameters are made available for download.