Model training

Training fine-tunes your student model on the generated dataset, then evaluates both the untrained and the trained student on your test set, which gives you the comparison with the base student and the teacher.

Before you start

An empty train.jsonl stops the run, so build the train set first. An empty test.jsonl produces a model with no scores.

The settings that matter are in base and tuning (Config file):

• base.student_model_name. The model you’re training. Start with Qwen3.5-4B, and go smaller only when the deployment target needs it (Supported models). Qwen3.8-27B trains only with tuning.use_qlora: true.
• tuning.per_device_train_batch_size (default 1).
• tuning.num_train_epochs (default 4).

For a reasoning student, see Reasoning models.

Smoke run: check memory

A smoke run checks that a student fits in memory before the full run spends a training credit. It trains one epoch on the 128 longest training rows and the 32 longest test rows (chosen by length, not per class), with the rest of your config unchanged. Run one per student, at the batch size the full run will use, since students of different sizes fit differently.

Download the dataset’s config into sweep. It can be read once the last expand job status reads Success:

distil dataset download-metadata -d sweep <dataset-id>

Copy sweep/config.yaml once per student, for example to sweep/student-qwen3.5-4b.yaml, and set base.student_model_name in each copy. Submit a smoke run with each copy. The command prints the new SLM’s id:

distil slm create-from-dataset --smoke --config sweep/student-qwen3.5-4b.yaml <dataset-id>

A smoke shows whether the run fits in memory, whether the training loss decreases in the log, and whether base and tuned metrics come back. Its scores come from one epoch on a small subset and say nothing about the full run. When it runs out of memory, Dealing with out-of-memory failures lists what to change.

Each smoke spends a slms_from_datasets_smoke_post credit, never a full training credit.

Full run

The full run uses all the training data and the default 4 epochs, with the settings the smoke fit in memory with. Submit it with the same config file; the command prints the new SLM’s id:

distil slm create-from-dataset --config sweep/student-qwen3.5-4b.yaml <dataset-id>

Check its status with that id until it reads Success:

distil slm status <slm-id>

Each run spends one slms_from_datasets_post credit, and new accounts start with five. To compare students, submit one run per student config against the same dataset; the submissions run concurrently and nothing is regenerated.

Each config file replaces the dataset’s config whole, and anything you leave out takes its default, so take a full copy of the downloaded config for each run (How the platform works).

Reading the results

Read the scores of the base and the tuned student:

distil slm metrics <slm-id>

Download the per-example predictions:

distil slm download-predictions <slm-id>

This writes <slm-id>-slm-predictions.jsonl to the current directory. It holds the tuned model’s predictions only; the base model’s scores are in the metrics, and its per-example predictions are not available.

The primary metric is llm-as-a-judge-reference-free by default, and binary for classification (Primary metric per task). Three numbers are read together:

• base is the untrained student, the floor.
• tuned is your trained student.
• Teacher is the ceiling, from teacher evaluation.
• The production model, when it was evaluated, is the floor the student has to clear (Teacher evaluation).

How much of the base-to-teacher gap the student closed is one number:

closed = (tuned - base) / (teacher - base)

closed near 1 means the student does the task about as well as the teacher; near 0 means training added little over the base student.

Scores vary between runs of the same evaluation (Metrics). What to do with the verdict is on Model iterations.

Dealing with out-of-memory failures

Apply these in order. Each costs more speed or quality than the one before.

1. Lower per_device_train_batch_size. Halve it, down to 1.
2. memory_optimized_training: 1. Gradient checkpointing, which recomputes activations in the backward pass instead of storing them. Slower.
3. memory_optimized_training: 2. Level 1 plus activation offloading, which moves the stored activations to CPU RAM. Much slower.
4. use_qlora: true together with memory_optimized_training: 2. A 4-bit base model, about 3x less GPU memory.
5. Remove the longest ~1% of training rows, which drive peak memory. Download the dataset, remove the rows from train.jsonl, and upload the directory as a new dataset with distil dataset create --data. The download is free; the upload spends a datasets_post credit.

The first three are config overrides; only the fourth changes the data.

To find the cause of a failed run, print its log:

distil slm logs <slm-id>

Search the log for the first error, not the last one. An out-of-memory crash often ends with a second, unrelated error, thousands of characters after the OutOfMemoryError that caused it.

What can be changed

All of these are config overrides on the same dataset, and nothing is regenerated:

• The student (base.student_model_name). A larger student has more capacity to learn the same data, at a higher serving cost. A different family can suit a task better at the same size.
• How much it trains (tuning.num_train_epochs, tuning.learning_rate, tuning.per_device_train_batch_size). More optimizer steps fit the training data more closely; at a fixed epoch count, a larger batch means fewer steps.
• The LoRA capacity (tuning.lora_r, tuning.lora_alpha_multiplier). How much of the model training can change. Deployment accepts only some lora_r values (Config file).
• Evaluation length (tuning.max_completion_length). How long an answer may be before it is cut off in evaluation.

Next

Serve it, either behind an inference endpoint or on your own machine. To change something and train again, see Model iterations.