4-Bit Floating Point

Computers store every number as a sequence of zeros and ones. Each such position is called a bit, and the more bits used, the more precisely a number can be captured. 4-Bit Floating Point is a format that uses only four of these positions. This makes exactly 16 different values representable, no more. The addition Floating Point means that these 16 values are not evenly distributed: near zero they lie close together, further out they spread further apart. This resembles a ruler whose markings are fine near the zero point and coarse toward the end.

Why four bits are suddenly said to be enough

Large language models consist of billions of numbers, known as parameters. They contain everything the model learned during training. In the past, each of these numbers was stored with 32 bits, today often with 16 bits. Anyone going down to 4 bits needs only a quarter to an eighth of the memory for the same model.

This determines what hardware is needed at all. A model with 70 billion parameters occupies around 140 gigabytes in 16 bits and thus does not fit on a single graphics card. In 4 bits it is about 35 gigabytes, and that runs on a single accelerator. A server rack thus becomes a single device.

Then there is speed. A large part of computing time is not spent on the actual calculation but on shuffling numbers back and forth between memory and the compute unit. Smaller numbers mean less transport overhead, less waiting time, and less power consumption. This is exactly why FP4 numbers appear in almost every announcement of new AI chips.

How 16 values are spread across billions of parameters

The four bits are divided up: one indicates the sign, two or three determine the order of magnitude, and the rest the fine gradation. A common format is called E2M1 and can represent values such as 0, 0.5, 1, 1.5, 2, 3, 4, and 6, each also negative. There are no further intermediate steps. A learned number like 1.37 is thus rounded to 1.5.

On its own this would be far too coarse. The trick is called scaling: the parameters are divided into small groups, say 16 or 32 at a time, and for each group an additional, more precise factor is stored. The 4-bit number then only indicates how large the value is relative to its group. This way each group can have its own scale, and the error stays small.

This after-the-fact conversion of a finished model is called quantization. It is important to distinguish it from the similarly named INT4: there the 16 values are evenly distributed, whereas with FP4 they are not. Because the numbers in AI models mostly lie close to zero, the uneven grid fits better. A common misconception, by the way, is that the model becomes smaller in the sense of dumber. It retains all its parameters; they are just recorded more coarsely.

FP4 in chip announcements and on your own computer

The format is most commonly encountered in product announcements from chip manufacturers. When Nvidia advertises multiplied performance for the Blackwell generation, this figure often refers to FP4 computations. Comparing it with older figures in 16 bits makes the leap sound bigger than it actually is. It is always worth checking which number format is meant in such statements.

The technology becomes practically noticeable in models meant to run on ordinary hardware. Freely available language models are usually offered in several precision levels, and the heavily quantized variants run on a good gaming PC or a current laptop. In 2025, OpenAI released its open models directly in a 4-bit format. Voice assistants that work on phones without an internet connection also rely on such heavily shrunk numbers.