Multi-Turn Tool Calling Explained
Multi-turn tool calling trains a model to produce the next function call given the whole conversation so far, rather than one call per isolated request. On distil labs it’s the chat-completion task, or chat-completion-agentic when the model also reads what the tools returned. It’s what you need when a user can say “actually, make that Tuesday”.
What makes tool calling multi-turn?
The input is a conversation, not a request. In single-turn tool calling each example is independent: one user message in, one call out. In multi-turn, the model receives alternating user and assistant turns and predicts the next assistant turn in that context.
That changes what the model has to learn. It must resolve references to earlier turns (“cancel that one”), track what has already happened (“you’re already in the backup directory”), and notice when the user has changed direction mid-conversation. None of that is inferable from the current message alone.
Single-turn and multi-turn are the same task type with different row lengths. Refer to the task selection guide.
Chat completion or agentic chat completion?
Both train on whole conversations with tool calls in them, declared under tools in job_description.json. They differ in what follows a tool call.
| Aspect | chat-completion |
chat-completion-agentic |
|---|---|---|
| After an assistant call comes | The next user turn | A tool turn carrying the result, then the model acts on it |
| The model sees tool results | Never | Yes, and decides whether to call again or answer |
tools in the job description |
Optional | Required |
| Typical shape | A command router behind an orchestrator that executes calls and talks to the user | A research or support agent that looks something up before answering |
Pick chat-completion when a deterministic orchestrator executes the call and the model never needs the result to decide its next move. Pick chat-completion-agentic when the next decision depends on what came back. The chat completion and agentic chat completion guides document the row rules for each.
How does the training data look?
The whole conversation lives in one messages array per JSONL line. The tools list uses OpenAI function schemas, the emitted arguments is a real JSON object rather than a stringified blob, and each assistant turn carries at most one call unless you raise synthgen.max_tool_calls_per_turn. Assistant turns may carry text next to the call, so “let me check that” and the lookup can be one message.
By default the platform expands every conversation into one example per assistant turn, for training and for evaluation. A four-call conversation therefore contributes four scored predictions rather than one, which is what you want, because it measures every decision point instead of only the last. base.should_expand_dataset in the config file controls it; set it to false for rows that are already split per turn.
Why does per-turn accuracy matter so much?
Because a conversation succeeds only if every turn succeeds, so per-call accuracies multiply. Single-turn benchmark numbers don’t show that compounding, so they read better than the system will behave.
Our FunctionGemma study worked the arithmetic out on base-model scores:
| Task | Base per-call | Implied 2-turn | Implied 5-turn |
|---|---|---|---|
| Smart home control | 38.82% | ~15.1% | ~0.9% |
| Banking voice assistant | 23.35% | ~5.5% | ~0.07% |
| Shell commands (Gorilla) | 9.90% | ~1.0% | ~0.001% |
The same arithmetic runs in your favour after training. At 80% per call a five-turn conversation completes about 33% of the time; at 95% it reaches 77%. The distance between “usually right” and “reliably right” is much larger than the headline numbers suggest, which is why fine-tuning isn’t optional here. The same 270M model card says as much: it “is intended to be fine-tuned for your specific function-calling task, including multi-turn use cases”.
After fine-tuning, that study’s students reached 90.9% to 96.7% per call, matching or exceeding their 120B teacher.
When is single-turn data enough?
When every request is genuinely self-contained. Three tests:
- No anaphora. If users never say “that one”, “the same again”, or “actually”, the conversation history carries no information.
- No state. If the correct call doesn’t depend on what already happened, history is noise.
- One call per session. A command bar or a webhook dispatcher is single-turn by construction.
When those hold, write single-turn rows: the data is simpler to write, and each example is independent so seeds are easier to collect. Otherwise write whole conversations, and note that training on single-turn rows won’t get you multi-turn behaviour. The model never sees a history, so it never learns to read one.
Whenever tools are declared, chat completion is restricted to the students and teachers marked for tool calling in the supported models catalog. During synthetic data generation, add a mutator with one value per tool so rare tools aren’t skipped (synthetic data with tools).
Related terms
Conversation expansion is the per-turn scoring above, controlled by base.should_expand_dataset. Tool result turn is a tool-role message carrying the output of the preceding call, which only chat-completion-agentic rows contain. Orchestrator is the deterministic loop that executes the call and feeds the result back. Related reading: what is tool calling for the single-turn case, FunctionGemma 270M and LFM2.5 350M for the two smallest viable students, and voice assistant command routing for a worked recipe.