Logit Space
The logit space is the intermediate stage in which an AI model assigns a raw score to every possible response option before these are turned into probabilities. These unrefined numbers reveal a great deal about how confident a model is and how it arrives at its output.
A language model writes text by repeatedly guessing the next piece of a word. Before it decides, it assigns a score to every possibility across the entire vocabulary. For a typical model that is around 100,000 numbers at once, one for each possible next building block. These raw scores are called logits, and the totality of all these numbers is called the logit space. They are not yet probabilities: they can be arbitrarily large or even negative, and together they do not add up to 100 percent. Only a conversion step turns them into percentage values, from which the next word is then drawn.
What the raw scores reveal about the model
Anyone who only sees the finished answer of a chatbot sees the result of a selection, not the selection itself. The logit space, by contrast, shows which alternatives narrowly lost out. If the runner-up trailed by only a hair, the decision was almost a coin toss. If the gap was huge, the model was practically certain. This information is lost as soon as only the selected text remains.
This is precisely why the logit space is so sought after in research. From it, measures of a model’s uncertainty can be derived. Such measures help detect hallucinations, that is, freely invented claims that sound confident. A model that is internally wavering between five candidates for a year number should be treated differently from one with a clear favorite.
There is also an economic side to this. Many providers only release the finished words through their interface, not the complete logits. The reason is not just data volume: from many logit outputs, conclusions can be drawn about a model’s internal structure. Competitors could use this to train their own model on the responses of the foreign one. The logit space is thus also a trade secret.
From number vector to word choice
At the end of the model sits a final computational layer. It takes the internal representation of the text so far and multiplies it by a large table. For every piece of vocabulary, a number falls out of this. This number roughly measures how well the word piece fits the context so far. You can imagine this like a jury awarding points, where each candidate is evaluated individually, without regard to the others.
After that comes a function called Softmax. It converts the scores into percentage values that add up to 100. Large differences are amplified in the process, small ones compressed. A control called temperature determines how strong this effect turns out. A low temperature divides the logits by a small number and makes the favorite even more dominant, so the model then answers predictably and dryly.
A high temperature evens out the differences and lets even unlikely words get a chance. This appears more creative but increases the risk of nonsense. A common misconception is to already regard logits themselves as probabilities. A logit of 12 means nothing on its own; only the distance to the other logits in the same step counts.
Where logits show up in practice
Anyone working with developer tools quickly encounters settings like logprobs or logit_bias. The first shows the most likely alternatives for every word piece output. The second allows individual word pieces to be artificially favored or suppressed by shifting their logits. This can, for example, force a model to answer only with yes or no.
The logit space also plays a role in shrinking models. In so-called distillation, a small model learns not just the correct answers of a large one, but its entire distribution of scores. The information that cat narrowly beat dog is more instructive than merely the word cat. News stories about supposedly copied models almost always revolve around this technique.
In everyday life, you encounter the logit space indirectly, for instance in the word prediction of a phone keyboard. The three suggestions above the keyboard are nothing other than the three highest scores of a small language model. Spam filters and image classifiers also work with the same logic: first raw scores per class, then the conversion into percentages.