What Does It Cost to Fine-Tune a Small Language Model?
The first two runs are free. Beyond that, the distil labs pricing page currently lists training credit packs at $1,000 for 10 runs, and prices production deployment separately. There’s no published per-hour or per-token training rate. A run is the billing unit, and volume pricing is quoted rather than listed.
What does distil labs publish for training cost?
Three things, and it’s worth being precise about which is which.
| Tier | Training cost | What it includes |
|---|---|---|
| Free | $0 | 2 full training runs, downloadable weights, self-hosting for R&D |
| Credit pack | $1,000 for 10 runs (currently listed) | Additional runs without a production plan |
| Enterprise | Custom | Training and deployment inside your own cloud, VPC or air-gapped |
That works out to about $100 per run at pack rate, which matches the voice assistant write-up. It puts training a banking assistant with a defined intent taxonomy and 50 example conversations at under $100.
Two caveats. The pricing page says “currently”, so treat the pack rate as a snapshot and check the page. And the model training documentation publishes no cost figure at all. It describes a run as “typically several hours” and says nothing about price, so anyone quoting you a per-run dollar amount from the docs is inventing it.
What is included in a run?
The whole pipeline, not just the gradient steps. This matters because it’s why “cost per training run” is the unit rather than cost per GPU hour.
- Teacher evaluation. The teacher answers your test set once, so you can decide whether to proceed.
- Synthetic data generation. The teacher is called repeatedly until the target dataset exists.
- Validation. Generated examples are checked for quality and deduplicated.
- Student fine-tuning. Supervised training on the generated set.
- Evaluation. The finished student is scored against the held-out test set.
Stage 2 is usually the largest consumer, and it’s teacher inference rather than student training. Most of what you pay for is the big model generating data, not the small model learning.
What drives the cost of a run?
Configuration, and mostly one parameter. Every value below is a documented default from the config reference.
| Parameter | Default | Effect on what a run consumes |
|---|---|---|
generation_target |
10,000 | Teacher calls scale with it, the dominant lever |
generation_in_single_call |
4 | Higher packs more examples per teacher invocation |
num_train_epochs |
4 | Roughly linear, but only in the fine-tuning stage |
teacher_model_name |
n/a | Larger MoE teachers cost more per generated example |
use_lora |
true |
Trains adapters instead of all weights |
rlvr_dataset_size |
0.0 |
Above zero adds a whole reinforcement learning stage |
At defaults, a 10,000-example target generated 4 at a time is on the order of 2,500 teacher invocations. Halve the target and you roughly halve the expensive stage. Enabling RLVR adds a stage that runs after supervised fine-tuning and calls an LLM judge for reward signals, so it isn’t a marginal change.
Does a bigger student cost more to train?
Yes, but less than you’d expect, because the student isn’t where the compute goes.
Two things suppress the effect. Adapter-based training only updates a small set of parameters: LoRA freezes the base model and trains low-rank matrices instead, and QLoRA goes further by holding the base in 4-bit while adapters stay in higher precision. Its authors fine-tuned a 65B model on a single GPU in 24 hours. And the synthetic data generation stage is identical regardless of student size.
So moving from a 1B student to a 4B one changes the fine-tuning stage, not the generation stage. What size model do you need covers choosing, and LoRA vs full fine-tuning covers the adapter trade-off.
What should you ask for if you need a firm number?
Ask for a quote against your actual task, and bring three things: your task type, your seed data volume, and your expected retraining cadence.
A single run is a one-off, but production models are retrained as data drifts, so the number that belongs in a business case is runs per year rather than runs. Calculating the ROI of a task-specific model folds that in.
The cheapest way to get a real figure is to spend nothing first: the free tier’s two runs give you downloadable weights and a measured accuracy number, which is enough to decide whether to buy more. For anything at volume, on-premise, or air-gapped, pricing is custom, so get in touch. And if you have no GPU of your own at all, fine-tune an LLM without a GPU describes the managed path these figures apply to.