Short definitions of the terms that come up repeatedly when training and deploying small language models. Each entry is a page of its own with enough context to be useful, and links to the longer treatment where one exists.
The terms fall into four rough groups.
How a model produces text. Before a model emits a token it produces logits, raw scores over the whole vocabulary. Temperature and sampling decide how those scores become an actual choice, which is why the same model can be deterministic or creative depending on one number. Tokenization is the step before all of this: text becomes integers, and the tokenizer’s vocabulary determines how efficiently your particular text is represented.
What limits a model at inference time. The context window bounds how much the model can attend to at once. The KV cache is what makes generation tractable. Without it, every new token would require recomputing attention over the entire sequence, and it’s also the reason memory use grows with conversation length rather than staying flat. Quantization reduces weight precision to trade a little accuracy for a lot of memory.
What changes during training. LoRA rank sets how much capacity an adapter has to learn your task, and it is the parameter people most often ask about. Epochs, batch size, and learning rate control how the optimisation proceeds. Catastrophic forgetting is the failure mode where training on your task erases capabilities the model previously had.
How data is organised. Held-out test sets, train and test splits, and the seed examples that synthetic generation expands from.
Why these particular terms
The selection is biased toward terms that change a decision. Knowing what a logit is matters because it explains why temperature works the way it does, and why a classification task can be scored on confidence rather than just the final answer. Knowing what the KV cache is matters because it explains why your memory ceiling depends on conversation length, which in turn constrains which model fits on a given device.
Terms that are interesting but don’t change what you would do (attention head counts, positional encoding schemes, the specific optimiser variant) are left out. There are better places to read about those, and knowing them won’t help you pick a student model or debug a bad training run.
On reading these
Definitions here are written for someone training a model, not for someone studying machine learning. Where a precise technical definition and a useful working one differ, these lean toward the useful one and say so. Where a term is commonly used loosely (“fine-tuning” and “distillation” get swapped constantly) the entry says what the distinction actually is.
Terms with a full article elsewhere on this site link to it. LoRA rank and catastrophic forgetting both have longer treatments in the fine-tuning cluster; the glossary entries are the short version for when you just need the definition.