Task selection

The task type is the first thing you choose and the hardest to change later. It decides the shape of your training rows, which metric your model is scored on, and which student and teacher models you can use. Every task type can be trained from production traces as well as from a labeled dataset.

The value goes in base.task in your config.yaml:

base:
  task: question-answering

Choosing between them

The output is	Choose	`base.task`
Free text for a single input: answers, extractions, transformations, JSON documents	Question answering	`question-answering`
One label from a fixed set	Classification: question answering with a mutator over the labels (Classifying into labels)	`question-answering`
A conversational turn: free text, tool calls, or both; the model never sees tool results	Chat completion	`chat-completion`
An agentic loop: tool calls whose results feed back, ending in an answer	Agentic chat completion	`chat-completion-agentic`

question-answering covers any text-in, text-out problem, labels included. When the input carries a document or retrieved text, put that text inside the user message. Data with tool results in it is chat-completion-agentic.

Chat completion with tools declared works with only some students and teachers, so check Supported models before you choose it.

Question answering

question-answering

The model takes one input and returns one free-text answer. The input carries everything needed to answer, including any document or retrieved passage.

Scored by an LLM judge on llm-as-a-judge-reference-free, so llm_as_a_judge_instructions in your job description decides what counts as a correct answer.

Use cases. Pulling the termination clause out of a contract, the total due off an invoice, the decisions from meeting minutes, answering from a retrieved passage.

Question answering data preparation →

Classifying into labels

Classification is a question-answering task whose answer is one label from a fixed set. There is no separate task type for it: the labels live in the job description and a mutator spreads the generated data over them.

Rows. The user turn holds the input text, the assistant turn the label and nothing else.
task_description. The labels, a definition of each, and “Answer with the label only.” This is the only place the teacher and the student read the labels from.
One mutator over the labels, one value per label. Each value completes the sentence “Generated examples should be …” (Mutators):
```
synthgen:
  mutators:
    - name: label
      description: which label the message has
      values:
        - "a message whose correct label is billing"
        - "a message whose correct label is technical_issue"
```
Generation is then spread roughly evenly over the labels. method: adaptive evens it further, at the cost of a teacher call per surviving row it labels. Set target_distribution: match_seed to follow the label mix of your training rows instead. One mutator with a value per label is the normal exception to the advice of 1-2 mutators of 3-10 values.
The metric. binary exact match is the accuracy (Metrics).

Nothing checks that every label is present in a split or that an answer is one of the labels, so count the labels yourself: in test.jsonl before you upload, and in the generated rows after the smoke run. For per-label error counts, group the rows of the predictions file by label.

Use cases. Intent detection for customer service, content moderation, sentiment analysis, topic categorization, triaging support tickets by department.

A config.yaml with task: classification from an older project is rejected; see Migrating for the changes.

Chat completion

chat-completion

A multi-turn conversation where assistant turns are free text, tool calls, or both. The model chats with the user and calls a tool when one helps. Tools are optional: leave them out of the job description and you get a conversational model without tools.

The model never sees tool results in this task: an assistant turn is always followed by the next user turn. If your conversations feed tool outputs back to the model, use agentic chat completion.

Scored by an LLM judge on llm-as-a-judge-reference-free, with tool-call metrics alongside it (Metrics).

Use cases. Customer support chat that occasionally looks something up, assistants that mix conversation with actions, replacing a general chat model that has a handful of functions.

Chat completion data preparation →

Agentic chat completion

chat-completion-agentic

Chat completion plus tool results: the model calls tools, reads the results fed back as tool messages, and continues with another call or a final text answer. Parallel calls in one turn are supported.

tools is required in the job description, and your data carries the full loop: assistant tool calls, tool results, and the closing answer. Scored like chat completion.

Use cases. Research assistants that search and then summarize, support agents that look up an order before answering, any agent whose answer depends on what a tool returned.

Agentic chat completion data preparation →

Collect traces with an endpoint, start from a trace file, or prepare a labeled dataset.