Activated Parameter Count

At its core, an AI language model consists of a very large number of numbers, the parameters. These numbers were set during training on example texts and determine which answer the model gives. You often read that a model has 700 billion parameters. That is the total number of all stored values. The activated parameter count says something different: it indicates how many of these values are actually used for a single query. In some modern models, this is only a few percent of the total.

Why the total number alone is misleading

For a long time, the total number of parameters was seen as a measure of a model’s strength. More parameters meant more knowledge, but also longer computation time and higher costs. This coupling has since been broken. A model can be enormous and still respond quickly if only a fraction of it factors in per query.

For costs, it is therefore primarily the activated number that matters. It determines how many computational steps are needed for an answer. Anyone running a service with millions of requests per day feels this directly on the electricity bill. Two models with the same total size can differ by a factor of ten in operating costs.

A common misconception is that fewer activated parameters also saves memory. That is not true. All parameters must reside in the memory of the graphics cards, including the ones currently inactive. What is saved is computation time, not space. That is why providers usually list both numbers side by side in their technical reports.

How a model switches off parts of itself

The most important mechanism behind this is called Mixture of Experts. The model is divided into many parallel building blocks, called experts. A small additional network, the router, selects only two or four of these building blocks for each word. All others remain silent during this computation step.

An example makes this tangible. Suppose a model has 256 experts and uses eight of them per word. Then all 256 reside in memory, but only about three percent actually compute. Together with the parts that are always active, this yields the activated parameter count. In well-known open models, this often lies between 20 and 40 billion, while the total count amounts to several hundred billion.

The comparison to a large library fits well here. The entire building must be constructed and heated, even if you only borrow three books. The effort of reading, however, depends only on those three books. The activated parameter count corresponds to the borrowed books, the total count to the entire collection.

The number in model announcements and price lists

When a company introduces a new language model, the specification is usually given in a notation like 671B total, 37B active. The B stands for billion. You find such specifications in trade articles, in model cards on platforms like Hugging Face, and in reports on Chinese and American model families.

For investors and industry observers, the number is an indicator of a provider’s computing costs. Declining activated parameter counts at constant quality mean that a model can be operated more cheaply. This is precisely how providers justify lower prices per request. Discussions about whether a model can run on a small number of graphics cards also revolve around this ratio.

In everyday life, you encounter this number indirectly. A chatbot that responds in fractions of a second very likely uses a model with few activated parameters. In smaller models without an expert split, the total count and the activated count are identical. Then the distinction disappears, and one simply speaks of the parameter count again.