Double Precision

A computer stores every number as a sequence of zeros and ones. The more such digits it reserves for a single number, the more precisely it can capture it. Double Precision means: 64 of these digits are used for a decimal number. The result is around 15 to 17 reliable digits, so roughly 3.141592653589793 instead of just 3.14. The name comes from the fact that 32 digits used to be considered normal precision, and 64 digits are simply double that amount. The format is usually abbreviated as FP64, where FP stands for the English term for a decimal number, floating point.

When small rounding errors become large

Every calculation with a limited number of digits rounds a little. In a single calculation this isn’t noticeable. It becomes a problem when millions of computational steps build on one another. Then tiny deviations add up and can render the final result unusable.

This is exactly what happens with weather models, fluid flow simulations, or calculations of satellite orbits. An error in the tenth decimal place grows over thousands of time steps into a visible difference. That’s why Double Precision in these fields is not a precaution but a requirement. Banks and physics labs also compute with 64 bits for the same reason.

But precision comes at a price. Twice as many digits means twice as much memory per number. And the computing units in graphics cards are considerably slower for this format than for shorter formats. On many AI accelerators, FP64 is ten to thirty times slower than the usual short format.

The three parts of a 64-bit number

The 64 digits are not simply one long number. They are divided into three sections. One digit indicates the sign, i.e. plus or minus. Eleven digits form the exponent, which determines how large or small the number is overall. The remaining 52 digits store the actual digits.

You can think of this like scientific notation. The number 6.022 times 10 to the power of 23 likewise consists of digits and an indication of magnitude. The computer does the same thing, just in binary. This allows both tiny and enormous numbers to fit into the same format. The range extends from about 10 to the power of minus 308 to 10 to the power of 308.

A common misconception is that Double Precision is exact. That’s not true. Even 0.1 cannot be represented cleanly in binary, no matter how many digits you use. FP64 merely makes the error so small that it no longer causes problems in most applications. Anyone who needs exact amounts, such as in accounting, calculates in whole cents instead.

Why AI chips use it sparingly

Double Precision comes up in almost every report about supercomputers. The well-known TOP500 list of the world’s fastest computers explicitly measures their performance in FP64 operations per second. A system is only considered powerful there once it can perform many of these precise calculations.

When training AI models, it’s the opposite. Neural networks tolerate imprecision surprisingly well, since their numbers are only approximate weightings anyway. That’s why they mostly work with 16 or even 8 digits. This saves memory and time and allows for larger models on the same hardware.

This explains a discussion that regularly appears in reports about graphics cards. Manufacturers significantly scale back or nearly omit FP64 units in pure AI accelerators. For chatbots this doesn’t matter, but for climate researchers it does. Universities and data centers therefore often buy different models than AI companies, even though both use the same chip generation.