What Is Tool Calling and How Do You Train for It?

Tool calling is a model mapping a natural-language request to a structured function call with valid arguments. On distil labs you train it with the chat-completion task: declare your tool schemas in the job description, give the platform a few dozen example conversations, and the student learns the mapping instead of looking it up.

What is tool calling in a small language model?

It’s constrained generation against a schema. The model doesn’t produce prose; it produces a function name and an argument object that must validate against the JSON Schema you supplied.

The tool catalogue is learned during training rather than retrieved at inference. You’re not passing documentation in the prompt for the model to read. The routing logic is in the weights. That’s what makes a 270M model a viable dispatcher: it doesn’t have to reason about tools it has never seen, only about the fixed set it was trained on.

The typical shape of the job: a fixed set of backend functions, a natural-language front door, and a deterministic orchestrator that executes whatever call comes back.

How do you train a model for it?

Two files carry the whole task. The chat completion data preparation guide has the full spec.

job_description.json holds task_description plus tools, a list of JSON Schemas in OpenAI’s function-calling format. Constrain aggressively here: enum on string parameters and minimum/maximum on numbers do real work, because they tell the teacher what an invalid call looks like.

train.jsonl holds the examples. Each is a messages conversation. For single-turn tool calling, that is one user turn and one assistant turn carrying the call in tool_calls:

{"messages": [{"role": "user", "content": "Oven heated to 220°C, pizza ready to bake."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "put_in_oven", "arguments": {}}}]}]}

An assistant turn may also carry text next to the call, or text alone, so the same task covers an assistant that says “let me check that” and fires the lookup in one message. Longer conversations are the same format with more turns; see multi-turn tool calling explained.

One format detail causes most data-prep failures: in tool_calls, arguments is a real JSON object. OpenAI’s chat-completions format stringifies it. The schemas in job_description.json follow the OpenAI shape; the calls in train.jsonl don’t. By default each assistant turn carries at most one call; raise synthgen.max_tool_calls_per_turn in the config if your rows contain parallel calls.

Which models can do tool calling on distil labs?

A subset, and this is enforced at validation rather than degraded quietly. Chat completion with tools declared is restricted to the models marked in the supported models catalog.

Role	Eligible when tools are declared
Students	Qwen3, Qwen3.5, Qwen3.6, Qwen3.8, Llama 3 family, LFM2/LFM2.5, FunctionGemma, Gemma 4
Teachers	Every teacher except the few marked as not tool-calling capable

Gemma 3 and SmolLM2 aren’t eligible students. A chat-completion job with no tools declared has no model restriction.

base:
  task: chat-completion
  student_model_name: Qwen3-1.7B
  teacher_model_name: zai.glm-5.3-low-thinking
evaluation:
  llm_as_a_judge_model_name: zai.glm-5.3-low-thinking
synthgen:
  teacher_temperature: 0.6

Synthetic data generation sees all the declared tools in every call and lets the teacher pick, so rare tools can end up missing. Add a mutator with one value per tool, such as "the conversation calls put_in_oven", so every generation call is asked for a specific one (synthetic data with tools).

How is a tool-calling model scored?

On call equivalence rather than text similarity. The metrics reference defines three tool-call metrics, alongside the judge metrics that score any text in the turn:

• tool_call_equivalence (recommended). Compares prediction and reference, treating unset parameters as their schema defaults.
• binary_tool_call. Strict name and argument equality, no default handling.
• staged_tool_call. Quarter-credit for one call on each side, correct function name, correct parameter keys, exact match. A 0.5 means the right function with wrong arguments.

Use the staged metric while iterating. It tells you whether your problem is schema adherence, routing, or slot filling, and those need different fixes.

How much does fine-tuning actually change?

Enough that base-model scores are close to irrelevant. Our LFM2.5 350M benchmark trained one 350M student on three tool-calling tasks against a 120B teacher:

Task	Teacher (120B)	LFM2.5 350M base	LFM2.5 350M tuned
Shell commands (Gorilla)	97.03%	61.4%	98.0%
Smart home control	92.11%	63.2%	96.7%
Banking voice assistant	96.95%	34.5%	95.9%

The student passed the teacher on two of three. That study used openai.gpt-oss-120b, the teacher those runs actually set.

Related terms

Multi-turn tool calling is the conversational case, and agentic chat completion is the task for a model that reads tool results. See multi-turn tool calling explained. Slot filling is extracting the arguments once the function is chosen. Orchestrator is the deterministic code that executes the returned call and asks for anything missing. For picking a student, see Qwen vs Llama vs Gemma for tool calling, LFM2.5 350M, and FunctionGemma 270M.