Temperature (sampling parameter)

Programs like ChatGPT write text word by word. Before each new word, the program calculates a probability for a great many possible continuations. After the sentence start “The sky is”, “blue” might get 60 percent, “gray” 20 percent, “purple” 0.1 percent. After that, a decision must be made about which word is actually taken. Temperature is the adjustment screw for exactly this decision. It determines whether the most probable suggestion nearly always wins or whether improbable words also get a real chance.

Between boredom and nonsense

Temperature decides the character of a text. At a very low value, around 0, the program practically always chooses the front-runner. The result seems factual and, given the same question, is almost identical every time. This is exactly what one wants for translations, summaries, or generated program code.

At high values from about 1.2 upward, the behavior tips over. The text becomes livelier, but also more error-prone. Unusual words slip through more often, and at some point the coherence disintegrates into gibberish. For advertising copy, brainstorming, or poems, a somewhat elevated value can nevertheless be useful.

An important distinction is often muddled. Temperature does not make a model smarter or dumber. The knowledge in the model remains exactly the same. What changes is only how boldly the selection is made from the already calculated possibilities. Another common misconception is that temperature 0 protects against false statements. A false claim with high probability is then even output with particular reliability.

What the number does to the probabilities

Technically, temperature is used as a divisor. The model first generates a raw value for each possible next word, a kind of score. These scores are divided by the temperature before being converted into percentage values. If you divide by a small number, the gaps between the candidates become larger. The favorite pulls away, the outsiders fall into the abyss.

With a large temperature, the opposite happens. The scores move closer together, and the distribution becomes flatter. 60 to 20 percent can thus become 35 to 28 percent. Chance gains more weight. You can picture it like a volume knob for differences: quiet voices are either suppressed or turned up as well.

Temperature rarely works alone. It is often combined with top-p, a rule that only allows the most probable words up to a certain cumulative percentage. This way, variety remains possible while completely absurd words are excluded from the outset. In addition, there is a so-called seed, a starting value for randomness. With a fixed seed, the same random selection can later be reproduced exactly.

The dial in chat windows and interfaces

Anyone who uses an AI only through a normal chat interface usually doesn’t see the temperature at all. Providers set a medium default value there, often around 0.7 to 1.0. This is why, when asking the same question twice, one rarely gets exactly the same answer twice. This very effect irritates many users, but it is simply the built-in random component.

The parameter becomes visible to developers. Anyone who addresses a model via a programming interface specifies temperature as a number in the call. Chatbot construction kits and tools for testing input texts also show a corresponding slider. In companies, the value is set deliberately: a customer service answer should be reliable, a campaign idea may deviate.

Temperature also appears in technical articles and model cards. When researchers compare results, they state the value used, because otherwise no one can retrace the measurement. Reports about strange outputs from a chatbot also occasionally mention it. A value set too high is a known explanation for answers that suddenly seem incoherent.