Learn
Practical guides to fine-tuning, distillation, and deploying small language models.
Calculating the ROI of a Task-Specific Model
A five-step procedure for turning a task-specific model into a break-even volume, using real CLI commands and a free evaluation gate before you commit any budget.
Deploy an SLM with Ollama
Loading a fine-tuned small language model into Ollama from a local GGUF or straight from Hugging Face, writing a Modelfile that preserves the training-time system prompt, and the chat-template trap specific to this runtime.
Deploy an SLM with vLLM
Serving a distilled small language model on a GPU with vLLM — environment setup, the tool-calling parser flags, querying the OpenAI-compatible endpoint, and the throughput and memory figures we measured on a single H100.
Fine-Tune a Model With 20 Examples
The minimal-dataset route end to end: four files, twenty labelled examples, and the distil CLI commands that turn them into a trained small language model.
How to Choose a Teacher Model for Distillation
Choose a teacher on measured accuracy for your own task, not parameter count. Criteria table, tool-calling constraints, and the teacher evaluation step that settles it.
How to Deploy a Fine-Tuned Small Language Model
The four ways to put a trained SLM into service — managed endpoint, local llama.cpp, self-hosted vLLM, and an offline download — with the constraint that selects each one and the mistakes that quietly break accuracy.
Run a Fine-Tuned SLM with llama.cpp
Serving a distilled small language model through llama-server, including the exact flags the distil CLI passes, why the Jinja chat template matters, and how to confirm the endpoint is serving your model and not a default one.
Running a Small Language Model on CPU
CPU inference for a distilled SLM: how to size the model to available RAM, which quantization to run, how to set thread counts, and the published tokens-per-second figures that tell you whether your workload is realistic.
Three Ways to Get Training Data for a Small Language Model
A decision guide to the three routes for getting training data onto the distil labs platform — live traffic collection, uploading existing traces, or a minimal hand-written dataset.