What Are Logits?
Logits are the raw, unnormalised scores a language model assigns to every token in its vocabulary at each step. They’re the model’s actual output, and the probabilities you see are those logits pushed through a softmax. Every sampling rule, from greedy decoding to temperature, is a way of turning that score vector into one token.
How does a logit become a token?
Through two steps, and only the second one is random.
| Stage | What it holds | Range |
|---|---|---|
| Logits | One raw score per vocabulary entry | Any real number |
| Softmax output | Probabilities over the same entries | 0 to 1, summing to 1 |
| Sampled token | A single integer id | One value |
Softmax exponentiates each score and divides by the total, so only the differences between logits matter, and adding a constant to all of them changes nothing. That’s why scaling logits by a divisor works as a creativity dial, which is the mechanism behind temperature.
Why do logits matter for distillation?
Because the original formulation of distillation trains on them rather than on the teacher’s final answer. Hinton, Vinyals and Dean’s Distilling the Knowledge in a Neural Network showed that a teacher’s softened output distribution carries more information per example than a hard label: the relative scores it gives to the wrong answers say something about how the teacher sees the problem.
Modern text distillation, including the pipeline behind a distil labs run, usually works from generated text rather than logit vectors, but the intuition survives. See knowledge distillation explained for how the two approaches relate.
Where do you actually meet them?
Anywhere you need confidence rather than an answer. A classifier’s logits over its label tokens give you a ranking and a margin, which is what lets you route low-confidence cases elsewhere instead of trusting every prediction. It’s worth pairing with the scoring rules in the metrics reference.
They’re also the reason a student model with a different vocabulary can’t simply copy a teacher’s output vector: the two index different token sets.