Overview

Whether you are looking to classify text, answer questions, interact with internal tools, or solve other language tasks, our step-by-step workflow will take you from initial concept to production-ready model. Let’s dive in!

Authentication

First, authenticate with the distil labs platform:

CLI

distil auth

API

import json
import requests

# See Account and Authentication for distil_bearer_token() implementation
auth_header = {"Authorization": f"Bearer {distil_bearer_token()}"}

Step 1: Create a model

Register a new model to track your experiment:

CLI

# Returns your model ID
distil model create my-model-name

API

import json
import requests
from pprint import pprint

response = requests.post(
    "https://api.distillabs.ai/models",
    data=json.dumps({"name": "my-model-name"}),
    headers={"Content-Type": "application/json", **auth_header},
)
pprint(response.json())
model_id = response.json()["id"]
print(f"Created model with ID={model_id}")

You can list all your models with:

CLI

distil model list

API

response = requests.get(
    "https://api.distillabs.ai/models",
    headers=auth_header,
)
pprint(response.json())

Step 2: Task selection and data preparation

Begin by identifying the specific task you want your model to perform. Different tasks require different approaches to data preparation and model configuration.

Learn more about task selection →

Step 2A: Upload traces

If you have production traces (logs of real interactions with an LLM), process them into training data instead of curating a dataset by hand. Traces are first uploaded as a PreparedTraces resource, then processed to produce training and test data (an Upload). The trace processing pipeline automatically filters, relabels, and splits your traces.

Your traces directory should contain:

File	Format	Required	Description
traces.jsonl	JSONL	Yes	Production traces in Langfuse or OpenAI messages format
job_description.json	JSON	Yes	Task objectives and configuration
config.yaml	YAML	Yes	Training and trace processing parameters

Upload your traces, then process them:

CLI

# Step 1: Store the trace files as a PreparedTraces resource
distil traces upload --data ./traces
# Output: Prepared traces created. ID: <traces-id>

# Step 2: Process them into an Upload
distil upload create-from-traces <traces-id>
# Output: Processing started. Upload ID: <upload-id>

API

# Step 1: Create a PreparedTraces resource
# Get presigned S3 URLs for uploading trace files
response = requests.get(
    "https://api.distillabs.ai/staging-prepared-traces-s3-urls",
    headers=auth_header,
)
urls = response.json()

# Upload files to S3 using the presigned URLs
requests.put(urls["traces_jsonl"], data=open("traces/traces.jsonl").read())
requests.put(urls["job_description_json"], data=open("traces/job_description.json").read())
requests.put(urls["config"], data=open("traces/config.yaml").read())

# Register the uploaded files
response = requests.post(
    "https://api.distillabs.ai/prepared-traces",
    data=json.dumps({
        "traces_jsonl": urls["traces_jsonl"],
        "job_description_json": urls["job_description_json"],
        "config": urls["config"],
    }),
    headers={"Content-Type": "application/json", **auth_header},
)
prepared_traces_id = response.json()["id"]
print(f"PreparedTraces created. ID: {prepared_traces_id}")

# Step 2: Kick off trace processing to produce an Upload
response = requests.post(
    "https://api.distillabs.ai/uploads/from-prepared-traces",
    data=json.dumps({"from": prepared_traces_id}),
    headers={"Content-Type": "application/json", **auth_header},
)
upload_id = response.json()["id"]
print(f"Trace processing started. Upload ID: {upload_id}")

Processing takes several minutes. Poll distil upload status <upload-id> until it reports success, then hand the processed data to your model:

distil upload download <upload-id> --destination ./processed
distil model upload-data <model-id> --data ./processed

Learn more about trace processing →

Step 2B: Upload minimal dataset

If you don’t have production traces, prepare a small structured dataset with labeled examples instead. A training job requires the following files in a directory:

File	Format	Required	Description
job_description.json	JSON	Yes	Task objectives and configuration
train.jsonl	JSONL	Yes	20+ labeled examples, each a messages conversation
test.jsonl	JSONL	Yes	Held-out evaluation set
config.yaml	YAML	Yes	Training hyperparameters
unstructured.jsonl	JSONL	No	Text documents relating to your problem domain which we may use for synthetic data generation

Upload your data to the model:

CLI

distil model upload-data <model-id> --data ./data

API

data = {
    "job_description": {"type": "json", "content": open("data/job_description.json").read()},
    "train_data": {"type": "jsonl", "content": open("data/train.jsonl").read()},
    "test_data": {"type": "jsonl", "content": open("data/test.jsonl").read()},
    "unstructured_data": {"type": "jsonl", "content": open("data/unstructured.jsonl").read()},
    "config": {"type": "yaml", "content": open("data/config.yaml").read()},
}
response = requests.post(
    f"https://api.distillabs.ai/models/{model_id}/uploads",
    data=json.dumps(data),
    headers={"Content-Type": "application/json", **auth_header},
)
upload_id = response.json()["id"]
print(f"Upload successful. ID: {upload_id}")

Learn more about data preparation →

Step 3: Teacher evaluation

Before training your specialized small model, validate whether a large language model can accurately solve your task with the provided examples. If the teacher model can solve the task, the student model will be able to learn from it effectively. Learn about teacher evaluation →

CLI

# Start teacher evaluation
distil model run-teacher-evaluation <model-id>

# Check status
distil model teacher-evaluation <model-id>

# Or, working from the upload ID directly
distil teacher-evaluation create-from-upload <upload-id>
distil teacher-evaluation status <teacher-evaluation-id>
distil teacher-evaluation metrics <teacher-evaluation-id>

API

import time

data = {"from": upload_id}
response = requests.post(
    "https://api.distillabs.ai/teacher-evaluations/from-uploads",
    data=json.dumps(data),
    headers={"Content-Type": "application/json", **auth_header},
)
eval_job_id = response.json()["id"]
print(f"Started teacher evaluation with ID: {eval_job_id}")

# Poll for completion
running = True
while running:
    response = requests.get(
        f"https://api.distillabs.ai/teacher-evaluations/{eval_job_id}/status",
        headers=auth_header
    )
    status = response.json()["status"]
    if status != "JOB_RUNNING":
        running = False
    print(f"Evaluation status: {status}")
    time.sleep(10)

metrics = requests.get(
    f"https://api.distillabs.ai/teacher-evaluations/{eval_job_id}/metrics",
    headers=auth_header
)
print(f"Results: {metrics.json()['teacher_performance']}")

Optional: inspect the synthetic training data

Training generates synthetic examples from your upload as part of its pipeline. You can also run that generation on its own first, to see what the platform would train on before committing to a training run.

CLI

# Generate a training dataset from the upload
distil training-dataset create-from-upload <upload-id>

# Poll until it finishes
distil training-dataset status <training-dataset-id>

# Preview up to 20 generated rows, for free
distil training-dataset sample <training-dataset-id>

# Or download the whole bundle
distil training-dataset download <training-dataset-id>

API

data = {"from": upload_id}
response = requests.post(
    "https://api.distillabs.ai/training-datasets/from-uploads",
    data=json.dumps(data),
    headers={"Content-Type": "application/json", **auth_header},
)
training_dataset_id = response.json()["id"]

# Poll until the generation job finishes
response = requests.get(
    f"https://api.distillabs.ai/training-datasets/{training_dataset_id}/status",
    headers=auth_header
)
print(f"Generation status: {response.json()['status']}")

# Preview the generated rows
response = requests.get(
    f"https://api.distillabs.ai/training-datasets/{training_dataset_id}/sample",
    headers=auth_header
)
pprint(response.json()["rows"])

Training still runs from the upload, as in the next step — generating a dataset here does not replace that.

Step 4: Model training

Once your teacher evaluation shows satisfactory results, train your specialized small language model using knowledge distillation.

Understand the model training process →

CLI

# Start training
distil model run-training <model-id>

# Check status
distil model training <model-id>

API

data = {"upload_id": upload_id}
response = requests.post(
    f"https://api.distillabs.ai/models/{model_id}/training",
    data=json.dumps(data),
    headers={"Content-Type": "application/json", **auth_header},
)
slm_training_job_id = response.json()["id"]
print(f"Training started with ID: {slm_training_job_id}")

# Check status
response = requests.get(
    f"https://api.distillabs.ai/trainings/{slm_training_job_id}/status",
    headers=auth_header
)
pprint(response.json())

# Get evaluation results when complete
response = requests.get(
    f"https://api.distillabs.ai/trainings/{slm_training_job_id}/evaluation-results",
    headers=auth_header
)
print(f"Evaluation results: {response.json()}")

Step 5: Download your model

Once training is complete, download your model:

CLI

distil model download <model-id>

API

response = requests.get(
    f"https://api.distillabs.ai/trainings/{slm_training_job_id}/model",
    headers=auth_header
)
print(f"Model download URL: {response.json()}")

Step 6: Model deployment

Deploy your trained model locally or using distil labs inference for immediate integration with your applications.

Explore deployment options →

Local deployment

Use the distil CLI with llama-cpp as the inference backend:

distil model deploy local <model-id>

Once running, get a ready-to-run invocation script with distil model invoke:

distil model invoke <model-id>

This outputs a command using uv that you can copy and run directly:

uv run $PATH_TO_CLIENT --conversation '[{"role": "user", "content": "Your question here"}]'

For question answering models that require context, wrap it in a <context> tag (followed by a newline) inside the first user message:

uv run $PATH_TO_CLIENT --conversation '[{"role": "user", "content": "<context>Your context here</context>\nYour question here"}]'

Remote deployment

Alternatively, deploy your model on distil-managed remote infrastructure using the distil labs inference:

distil model deploy remote <model-id>

The CLI will provision your deployment and display the endpoint URL, API key, and a client script you can use to query your model.

Once deployed, you can also use distil model invoke to get a ready-to-run invocation script for your remote deployment:

distil model invoke <model-id>

Next steps

You’ve successfully trained and deployed a specialized small language model! For more details, explore:

• Tutorials for complete end-to-end examples
• Deployment options for production deployment