Model Hardware Standard

An AI model is a very large file containing billions of learned numerical values. For it to generate answers, this file must be loaded onto a computer and processed. Not every computer can manage this: some have too little memory, others chips that are too slow. A Model Hardware Standard is a binding description of what equipment is needed for a particular model to run properly. It states, for example, how much working memory the graphics chip must have and which types of computation it should support. Providers publish such specifications so that customers know in advance whether their devices are sufficient.

What goes wrong without clear requirements

Anyone wanting to deploy an AI model usually buys the hardware first. Compute chips for AI are expensive; individual servers quickly cost several tens of thousands of euros. A wrong purchase cannot simply be returned, because the devices are usually permanently installed. A clearly formulated standard prevents exactly this mistake, because it makes the decision possible before the purchase.

The second reason is predictability for companies. A company that integrates a model into its software wants to know whether it will still run on the same servers in two years' time. If a provider discloses its requirements, the company can plan its expenditure over several years. Without such information, all that remains is trial and error, which costs time and money.

Added to this is competition between chip manufacturers. If a model only runs on the chips of a single provider, a strong dependency arises. Standards that allow multiple types of hardware weaken this dependency. This is precisely why the topic is also of interest to investors and business journalists.

Which figures it contains

The most important figure is almost always the memory of the graphics chip. A model must fit entirely into this memory, otherwise it either fails to start at all or becomes extremely slow. As a rough rule of thumb, a model with eight billion learned values needs about sixteen gigabytes of memory. Larger models therefore only fit across several chips working together at once.

A second point is the precision with which the numbers are stored. Each value can be stored very precisely or rounded more coarsely. Coarser storage often halves the requirement and speeds up computation, but costs a little quality. A standard therefore also states which precision levels the chip must support.

Third, it concerns the connection between multiple chips and the software. If eight chips are working on one model, they must constantly exchange intermediate results. If this connection is too slow, the expensive chips spend most of their time simply waiting. In addition, there are drivers and program libraries in specific versions, without which the model will not start.

From the model card to the quarterly figures

The term is encountered most directly on so-called model cards. These are short data sheets that providers supply with every published model. Alongside the capabilities, they also list the recommended chips and the required memory. Anyone wanting to try out a freely available model on their own computer reads exactly this information first.

In the news, the topic comes up when a model maker and a chip manufacturer announce a partnership. Such announcements usually mean that a model has been officially approved for certain hardware. For the companies involved, these represent predictable revenues, which is why the stock market reacts to them. A common misconception here is confusing the standard with a legal regulation. It is almost always a technical recommendation from the manufacturer, not a rule set by a regulatory authority.

The principle also operates in everyday life without anyone noticing. When a phone converts voice recordings into text directly on the device, the model has previously been adapted to that phone’s chips. Older devices often do not get such features because they lack the necessary memory. This is precisely a Model Hardware Standard in its most visible form.