Parameter Count
The parameter count indicates how many adjustable numerical values an AI model has inside it. It is the most common measure of a model's size – but on its own says nothing about its quality.
A program for artificial intelligence consists at its core of a very long list of numbers. These numbers are called parameters. They are not typed in by humans, but are automatically set during learning from example data. You can imagine each parameter as a tiny knob that determines one detail of the behavior. The parameter count is simply the answer to the question: How many such knobs does this model have? In today’s language programs, that is often several billion to several hundred billion.
Why billions of knobs matter
The parameter count is the number companies use to compare their models. It appears directly in names: a model with the suffix “7B” has seven billion parameters, “70B” stands for seventy billion. Roughly speaking: more parameters mean more room to store knowledge and language patterns. Small models can solve simple tasks, but lose track more quickly with long texts and difficult questions.
At the same time, every parameter costs money. The numbers have to sit in the memory of the graphics cards while the model is working. As a rule of thumb, a model needs about two gigabytes of memory per billion parameters. A model with 70 billion parameters therefore no longer fits on a normal laptop, while a model with three billion does. This is exactly where it is decided whether an AI runs on your phone or only in a data center.
A common misconception is that more parameters are automatically better. A model with eight billion parameters carefully trained on good data often beats a poorly trained one with seventy billion. The amount and quality of the training texts counts at least as much as the size.
What a parameter actually does in the model
A model processes text by multiplying and adding numbers together. The parameters are the factors in these calculations. They determine how strongly a particular signal is passed on or weakened. During learning, each parameter is adjusted minimally countless times until the model, on average, makes good predictions.
A school-level analogy is the equation of a line y = m·x + b. There, m and b are two parameters that determine the position of the line. A language model follows the same principle, only with billions instead of two such values and across many layers in succession. That is why it can describe relationships that a single line could never capture.
It is important to distinguish between parameters that exist and parameters that are actually used. Some models are built so that only a fraction of their parameters compute per request. Such a model may have 400 billion parameters but activate only 30 billion of them. Such figures then appear side by side in technical reports as “total parameters” and “active parameters”.
The number in product names and stock market news
Anyone reading AI news constantly comes across this number. It appears in model names, in press releases, and in comparison tables. Providers use it as an argument for performance, critics as an indicator of power consumption and cost. It is notable that the largest companies now often keep the parameter count of their flagship models secret.
The number becomes practically relevant when downloading freely available models. There, next to each download, it states whether it is a version with three, eight, or seventy billion parameters. Anyone who wants to try them out on their own computer chooses based on their graphics card. The smaller variants respond faster, the larger ones more thoroughly.
The number is also indirectly interesting for investors. Larger models need more chips, more power, and larger data centers. The trend toward ever more parameters has therefore strongly influenced the revenue of chip manufacturers. Lately, however, development is moving in both directions: very large models for difficult tasks and very small ones for phones and cars.