What Is Text Classification with a Small Language Model?

Text classification is the task where a model reads an input and returns exactly one label from a fixed, known set. On distil labs it is a question-answering task whose answer is the label: you list the labels and their definitions in the task description, add a mutator with one value per label, and the trained model emits the label string and nothing else. No explanation, no hedging.

How does classification work on a fine-tuned small model?

The model is still generating text; the task description constrains what that text is allowed to be. A classifier trained this way outputs the label token sequence directly, which is why downstream code can consume it without a parser.

Three things make it work. The label definitions in task_description tell the teacher what each label means, so it can generate synthetic examples that respect the boundaries. A mutator with one value per label makes the teacher generate for every label in turn, so no label is rare in the synthetic set. And because the output space is closed, evaluation is unambiguous: the binary metric is an exact match on the label, with no scoring model in the middle.

Base models are bad at this out of the box, and predictably so. The failure modes are consistent: confusing overlapping categories, missing weak signals in short inputs, and producing an extra sentence around the label.

What does the training data look like?

Four files, of which two carry the task definition. Classifying into labels in the docs has the full spec, and the question answering data preparation guide covers the shared file rules.

File	Required	What it holds
job_description.json	yes	task_description with every label, a definition of each, and “Answer with the label only.”
train.jsonl	yes	Labelled examples, each a messages pair
test.jsonl	yes	Held-out examples, same shape
config.yaml	yes	base.task: question-answering, the model choices, and one mutator over the labels
traces.jsonl	no	Unlabelled in-domain conversations to steer synthetic generation

A training line is a two-turn conversation whose assistant turn is the bare label:

{"messages": [{"role": "user", "content": "Why is there a fee for getting cash?"}, {"role": "assistant", "content": "cash_withdrawal_charge"}]}

The config carries the mutator. Each value completes the sentence “Generated examples should be …”:

base:
  task: question-answering
  student_model_name: Qwen3.5-4B
  teacher_model_name: zai.glm-5.3-low-thinking
synthgen:
  mutators:
    - name: label
      description: which label the message has
      values:
        - "a message whose correct label is cash_withdrawal_charge"
        - "a message whose correct label is declined_cash_withdrawal"

The label strings in train.jsonl must match the labels in task_description exactly; nothing checks this for you, so count the labels in both files before you upload. The label definitions are worth more effort than the examples. They’re what the teacher reads when it fabricates thousands of new examples per label, and a vague definition produces vague synthetic data.

How well do small models classify after fine-tuning?

Well enough to match a teacher two to four hundred times their size, on published distil labs work.

Task	Student	Base	Fine-tuned	Teacher
Email labelling, 10 classes	Qwen3-0.6B	38%	93%	93% (GPT-OSS-120B)
AI-generated text detection, 2 classes	Gemma 3 270M	~40%	100%	100% (GPT OSS 120B)

Both studies used openai.gpt-oss-120b as the teacher, which is what those runs actually set; for new work the current pick is zai.glm-5.3-low-thinking. The email classifier started from 154 seed examples expanded to 10,000 synthetic ones; the slop detector started from about 50. The pattern in both is the same: the base model is near chance, and the entire gap is closed by fine-tuning, not by model size.

The second row needs a caveat. That 100% is on a small held-out set from one Kaggle dataset, and the same model dropped to roughly 95% after 4-bit quantisation and to 88% on formal human emails in real-world spot checks. A number that clean is a statement about the test set as much as the model.

When is classification the wrong frame?

When the label set isn’t actually closed, or when one input can legitimately carry two labels.

• Open-ended output. If what you need is a sentence rather than a category, use question answering with free-text answers instead.
• Multi-label problems. A classifier assigns one label per input. Overlapping labels need either a merged taxonomy or one binary classifier per label.
• Arguments, not just a name. If the downstream system needs parameters (an account number, a date), you want tool calling, where the output is a schema-checked call.
• A taxonomy nobody can apply consistently. If two annotators disagree on 20% of your examples, the model will inherit that disagreement. Fix the definitions first.

Related terms

Label definition is the per-label sentence in task_description; it doubles as the prompt the teacher uses during generation. Label mutator is the synthgen.mutators entry with one value per label that spreads generation evenly across them; see mutators. Seed data is your handful of hand-labelled examples before synthetic expansion. See few-shot fine-tuning from 10 examples. For choosing a student, Qwen3 1.7B vs Gemma 3 1B for classification compares the two most common candidates, and what size model do you need covers the size question generally.