Determinism

A calculator is deterministic. Anyone typing in 17 times 23 ten times gets 391 ten times. Determinism is exactly this property: same input, same conditions, always the same result. The opposite is called nondeterminism. There, the same question can yield a different answer the second time, even though nothing outwardly changed. Language programs like ChatGPT frequently behave this way.

Why repeatable answers are worth money

Anyone who wants to test software needs repeatability. A test only works if a bug reliably reoccurs. If it disappears on the second attempt on its own, it’s nearly impossible to track down. Programmers mockingly call such bugs Heisenbugs, because they vanish the moment you look at them.

In some industries, determinism is even a regulatory requirement. A bank must be able to explain why a loan was denied. If the same data yields a yes on Monday and a no on Tuesday, that’s hardly justifiable. Likewise, in medicine, tax calculation, or a car’s braking systems, no one wants chance to be involved.

That’s why nondeterminism is one of the main reasons many companies don’t simply integrate AI systems into existing workflows. You can’t test them the way you test normal software. Instead of fixed checks, you need sampling and evaluations, which is more effort and never provides one hundred percent certainty.

Where the randomness in language models comes from

A language model doesn’t predict a single fixed next word. It calculates a probability for many possible words. Then one is selected from this list, often chosen randomly weighted by the probabilities. A dial called temperature controls how strongly randomness comes into play.

If you set the temperature to zero, the most probable word is always chosen. In theory, that would make the model deterministic. In practice, this often isn’t enough. Graphics cards compute with limited precision, and the order in which many requests are processed simultaneously changes tiny intermediate results.

These tiny deviations can tip which word narrowly comes out on top. From that point on, the sentence goes in a different direction, since every word influences the next. A rounding error at the third decimal place thus turns into a visibly different answer. A second factor adds to this: providers continuously update their models, often without users noticing.

Determinism in everyday life and in products

You encounter this difference constantly in everyday life. A search engine usually delivers similar results for the same search. A chatbot, on the other hand, phrases the same explanation differently the second time. This feels more lively, but it’s a real problem as soon as you have to rely on the answer.

Developers are therefore familiar with the seed parameter. It sets the starting value for randomness and makes results at least approximately repeatable. Several AI providers offer such a setting, but explicitly do not guarantee exact equality. Tenders and contracts today often specify how much answers are allowed to vary.

A common misconception is confusing determinism with correctness. A program can give the same wrong answer every time and still be deterministic. Conversely, a fluctuating model can be right most of the time. Determinism only says something about repeatability, not about quality.