Non-deterministic system

A calculator is computable: 7 times 8 always equals 56, today as tomorrow. Such systems are called deterministic, because the input fully determines the result. A non-deterministic system behaves differently. It can produce different results for exactly the same input, without any error being present. This variation is part of the design and is intentional, or at least accepted. Many AI programs that generate text or images work exactly this way.

Why software suddenly stops being reproducible

Classic software was built for decades on a fixed promise: same input, same result. Testing relies on this. A program is run with a known input and the output is compared against the expected value. If they match, the test is considered passed. With a non-deterministic system, this procedure no longer works cleanly.

For companies, this is a practical problem. A bank that has a language model answer customer inquiries cannot guarantee that two customers asking the same question get exactly the same answer. Debugging also becomes harder. If an error doesn’t recur on the second attempt, it can barely be pinned down.

A common misconception is to equate non-determinism with unreliability. The two are not the same. A system can be factually correct in every single answer and still phrase it differently each time. Conversely, a strictly deterministic program can reliably output the same nonsense every time.

Where the randomness in the model comes from

A language model writes text word by word. For the next spot, it does not compute a single solution but a list of candidates with probabilities. After “The capital of France is”, “Paris” might be at 97 percent and a few other words share the rest. Which word is actually chosen is decided by a random draw from this list. That’s why the second run is rarely identical to the first.

How much randomness is applied is controlled by a dial called temperature. At temperature zero, the most probable candidate is always chosen, and the behavior becomes nearly deterministic. At high values, the model reaches for improbable words more often, and the text becomes more creative and riskier. Developers set the dial depending on purpose: low for data analysis, higher for advertising copy.

Even at temperature zero, some residual variation often remains. Graphics cards compute many partial results in parallel and add them up in varying order. With decimal numbers, this leads to tiny deviations in the last digits. If two candidates are very close to each other, this can sometimes tip the decision the other way. On top of that, providers update their models in the background.

Where variation is a nuisance and where it’s desired

In everyday use, the effect is immediately noticeable. Anyone who types the same prompt into a chatbot twice gets two similar but not identical answers. With image generators, the difference is even more pronounced: the same description produces completely different images. Some tools therefore display a so-called seed, a starting number for the randomness. Entering the same seed again often allows a result to be reproduced.

The term comes up in the business press when it concerns regulation and liability. Regulators demand that automated decisions be traceable. In medicine, credit lending, or applications to government agencies, a fluctuating system is hard to justify. That’s why companies often combine AI models with fixed rules that check and constrain the result at the end.

Conversely, the variation is sometimes exactly the point. Anyone who needs ten suggestions for a product name doesn’t want the same one ten times. When testing software, it also helps to run a model multiple times and compare the answers. If they diverge strongly, that’s a sign of an ill-defined task.