Using the REST API

Everything the CLI does, the REST API does. Use it when your work is already scripted, or when you would rather call HTTP than shell out to commands.

Both create the same entities on the same platform, accept the same config overrides, and identify each entity by the same UUID. A project can move between them freely, since an id from one works in the other.

The complete endpoint schema is at api.distillabs.ai/docs. This page is the working guide.

Prerequisites

pip install requests pyyaml
distil auth

The API is https://api.distillabs.ai. It authenticates with a bearer token from the CLI, so install the CLI and sign in first: Ways to use distil labs. distil access-token refreshes the session and prints a new access token each time it runs.

Access tokens are short-lived, so the helpers below fetch a token per request rather than holding one.

Helpers

Every example on this page assumes this block.

import json
import os
import subprocess
import time
from pathlib import Path

import requests
import yaml

PLATFORM_URL = os.getenv("DL_PLATFORM_URL", "https://api.distillabs.ai")
POLL_INTERVAL_SECONDS = 20
POLL_TIMEOUT_SECONDS = 6 * 60 * 60

# A staging response names each file by extension; a create body names it by
# field. These maps are the only place that difference is spelled out.
DATASET_FIELDS = {
    "config": "config_yaml",
    "job_description": "job_description_json",
    "train_data": "train_data_jsonl",
    "test_data": "test_data_jsonl",
    "traces": "traces_jsonl",
}
PREPARED_TRACES_FIELDS = {"traces_jsonl": "traces_jsonl"}
SLM_FIELDS = {"model": "model_tar", "config": "config_yaml"}


def auth():
    result = subprocess.run(["distil", "access-token"], capture_output=True, text=True)
    if result.returncode != 0:
        raise RuntimeError(result.stderr.strip())
    return {"Authorization": f"Bearer {result.stdout.strip()}"}


def raise_with_body(response):
    """raise_for_status hides the response body, which is where validation
    errors live. Print it before raising."""
    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError:
        print(f"{response.status_code}: {response.text[:2000]}")
        raise


def get(path):
    response = requests.get(f"{PLATFORM_URL}{path}", headers=auth())
    raise_with_body(response)
    return response.json()


def post(path, body):
    response = requests.post(
        f"{PLATFORM_URL}{path}",
        data=json.dumps(body),
        headers={"content-type": "application/json", **auth()},
    )
    raise_with_body(response)
    return response.json()


def stage(staging_route, files, fields):
    """PUT each local file to its presigned URL and return the create body.
    Only the files given are staged and named: a dataset file left out is an
    empty split."""
    urls = get(staging_route)
    body = {}
    for field, filepath in files.items():
        url = urls[fields[field]]
        requests.put(url, data=Path(filepath).read_bytes()).raise_for_status()
        body[field] = url
    return body


def config_of(collection, entity_id):
    """Fetch a parent's whole config as a dict. An override replaces the config
    rather than merging into it, so every override starts here. Costs nothing.
    For a dataset an expand produced, the URLs are null until its job finishes."""
    urls = get(f"/{collection}/{entity_id}/download-metadata")
    if urls["config_url"] is None:
        status = get(f"/{collection}/{entity_id}/status")["status"]
        raise RuntimeError(f"{collection}/{entity_id}: not available yet (status {status})")
    response = requests.get(urls["config_url"])
    response.raise_for_status()
    return yaml.safe_load(response.text)


def poll(collection, entity_id, timeout_seconds, status_field="status"):
    deadline = time.time() + timeout_seconds
    while time.time() < deadline:
        status = get(f"/{collection}/{entity_id}/status")[status_field]
        if status == "JOB_SUCCESS":
            return
        if status in ("JOB_FAILURE", "JOB_STOPPED"):
            logs = get(f"/{collection}/{entity_id}/logs")["logs"]
            raise RuntimeError(f"{collection}/{entity_id} {status}:\n{logs[-4000:]}")
        print(f"{collection}/{entity_id}: {status}", flush=True)
        time.sleep(POLL_INTERVAL_SECONDS)
    raise TimeoutError(f"{collection}/{entity_id} did not finish in {timeout_seconds}s")

Keep raise_with_body. Validation errors arrive in the response body, and raise_for_status() throws them away, so a 400 that names the exact row and field at fault reads as a bare HTTPError.

The entity model

Stage	Submit	Read
Traces	`POST /prepared-traces`, from a staged `traces.jsonl` or an endpoint’s name	`GET /prepared-traces[/<id>]`, `GET /prepared-traces/<id>/download`
Dataset	`POST /datasets` from staged files, or `POST /datasets/from-prepared-traces` with a config and a job description inline	`GET /datasets[/<id>]`, `GET /datasets/<id>/{status,logs,metrics,sample,download,download-metadata}`
Relabel traces, test or train	`POST /datasets/from-datasets[-smoke]` with `operation: relabel_traces_{test,train}`	same
Synthetic data generation, test or train	`POST /datasets/from-datasets[-smoke]` with `operation: generate_synthetic_data_{test,train}`	same
Teacher evaluation	`POST /teacher-evaluations` with `from_dataset_id`	`GET /teacher-evaluations[/<id>]`, `GET /teacher-evaluations/<id>/{status,logs,metrics,download-metadata}`
Model training	`POST /slms/from-datasets[-smoke]`, or `POST /slms` from a staged tarball	`GET /slms[/<id>]`, `GET /slms/<id>/{status,logs,metrics,download,download-metadata}`
Deployment	`POST /deployments/from-slms`	`GET /deployments/<id>/{status,endpoint,logs}`, `DELETE /deployments/<id>`

/seed-datasets and /training-datasets read old entities: list, single read, download and download-metadata. Their other routes, /prepared-traces/with-expanded-test-set and the status, logs, metrics and download-metadata routes of a traces object answer 410 Gone naming the replacement. A 410 means the path is the problem, not the payload. See Migrating old entities.

Every collection also has a list and a single read:

datasets = get("/datasets")                  # newest first
dataset = get(f"/datasets/{dataset_id}")     # one, by id

A list item has id, created_at and the parent’s id under its own key (dataset_id, slm_id). A dataset also carries its operation and either parent_dataset_id or parent_prepared_traces_id; a traces object its source (direct_upload or inference_endpoint). The list carries no status; the single read adds status (deployment_status and endpoint_status for a deployment), which a dataset that no job produced omits: read that as JOB_SUCCESS. Use the list to recover an id you’ve lost, and parent_dataset_id to walk a chain back to its root.

Staging files

Staging is a three-step exchange: GET /staging-<kind>-s3-urls returns a presigned PUT URL per file, each file is PUT to its URL, and those URLs are posted back as the create body. The stage() helper does all three.

The staging response names each file by extension (train_data_jsonl) while the create body names it by field (train_data). The helper hides that difference.

Staged bundles expire after seven days if they never become an entity.

Creating a traces object

body = stage("/staging-prepared-traces-s3-urls", {"traces_jsonl": "traces.jsonl"}, PREPARED_TRACES_FIELDS)
prepared_traces_id = post("/prepared-traces", body)["id"]

The create checks only that the file is there. A malformed trace fails the first expand that reads it, and the job log names the cause. An endpoint’s records go in the same route with inference_endpoint_name in place of traces_jsonl (Traces from an endpoint).

Creating a dataset

From a traces object, with the config and the job description inline as JSON objects:

dataset_id = post("/datasets/from-prepared-traces", {
    "from": prepared_traces_id,
    "config": yaml.safe_load(Path("config.yaml").read_text()),
    "job_description": json.loads(Path("job_description.json").read_text()),
})["id"]

No job runs: the traces are copied, train and test are empty, and the dataset is ready at once. The trace file’s shape must match trace_processing.observation_format in the config (Traces).

From files on disk (Dataset overview), staged file by file. config and job_description are required; stage any of train_data, test_data and traces, and leave out the ones you don’t have:

body = stage("/staging-datasets-s3-urls", {
    "config": "config.yaml",
    "job_description": "job_description.json",
    "train_data": "train.jsonl",
    "test_data": "test.jsonl",
    # "traces": "traces.jsonl",
}, DATASET_FIELDS)
dataset_id = post("/datasets", body)["id"]

POST /datasets validates the bundle and returns 400 with the validation error when it fails. That’s this backend’s dry run. The route is metered, but only a successful create spends a credit, so validating a broken bundle repeatedly is free. A 402 here means the balance was already zero before the bundle was read.

The expand operations

One route runs all four (The expand operations). The body names the parent and the operation; the new dataset’s parent_dataset_id is the parent and its operation is the one sent:

test_relabelled_id = post("/datasets/from-datasets", {
    "from": dataset_id, "operation": "relabel_traces_test",
})["id"]
poll("datasets", test_relabelled_id, POLL_TIMEOUT_SECONDS)

train_relabelled_id = post("/datasets/from-datasets", {
    "from": test_relabelled_id, "operation": "relabel_traces_train",
})["id"]
poll("datasets", train_relabelled_id, POLL_TIMEOUT_SECONDS)

train_generated_id = post("/datasets/from-datasets", {
    "from": train_relabelled_id, "operation": "generate_synthetic_data_train",
})["id"]
poll("datasets", train_generated_id, POLL_TIMEOUT_SECONDS)

The counts come from the parent’s config: trace_processing.num_{split}_relabelled for the relabeling operations, synthgen.{split}_generation_target for the generation ones. Errors: 400 for a config that fails validation, 404 for a missing parent, 409 for a parent whose job hasn’t succeeded, 402 for insufficient credits. A job whose output fails dataset validation (for example a row longer than synthgen.validation_max_total_length) ends in JOB_FAILURE with the cause in the log.

Submitting the rest of the pipeline

Teacher evaluation, then training:

teacher_evaluation_id = post("/teacher-evaluations", {"from_dataset_id": dataset_id})["id"]
poll("teacher-evaluations", teacher_evaluation_id, POLL_TIMEOUT_SECONDS)

slm_id = post("/slms/from-datasets", {"from": train_generated_id})["id"]
poll("slms", slm_id, POLL_TIMEOUT_SECONDS)

A sweep is N submissions against the same dataset, one per student. Read the dataset’s config once, then vary one field per submission; the deep copy matters, because mutating one dict across iterations would send every student the last one’s value:

base_config = config_of("datasets", train_generated_id)
slm_ids = {}
for student in ["Qwen3.5-4B", "Qwen3.5-2B"]:
    config = json.loads(json.dumps(base_config))
    config["base"]["student_model_name"] = student
    slm_ids[student] = post("/slms/from-datasets", {"from": train_generated_id, "config": config})["id"]
for student, slm_id in slm_ids.items():
    poll("slms", slm_id, POLL_TIMEOUT_SECONDS)

Smoke runs

The expand operations and training each have a -smoke route that runs the job small, on the same parent and with the same config otherwise:

smoke_dataset_id = post("/datasets/from-datasets-smoke", {
    "from": dataset_id, "operation": "generate_synthetic_data_train",
})["id"]
smoke_slm_id = post("/slms/from-datasets-smoke", {"from": train_generated_id})["id"]

The expand smoke forces the relabeled count or the generation target of the chosen operation to 128 whatever the config says, and writes a dataset that is a side branch of the parent: submit the full run on the same parent. The training smoke runs one epoch on the 128 longest train rows and the 32 longest test rows of the dataset, which makes it a memory check for the student and batch size the full run will use. All take the same config and job_description overrides as the full routes, and spend their own *_smoke_post credits rather than a full run’s. See How the platform works.

Overrides

Every job create accepts optional config and job_description objects inline alongside from (or from_dataset_id for a teacher evaluation): /datasets/from-datasets and -smoke, /teacher-evaluations, /slms/from-datasets and -smoke. There’s no staging step for either, and an omitted one is read from the parent dataset’s own files.

config = config_of("datasets", dataset_id)
config.setdefault("synthgen", {})["train_generation_target"] = 5000

generated_id = post("/datasets/from-datasets", {
    "from": dataset_id,
    "operation": "generate_synthetic_data_train",
    "config": config,
})["id"]

Each override replaces the parent’s file whole. Anything you leave out reverts to a library default rather than the parent’s value, so an override is always read-edit-resend, never hand-built. Two shapes to know:

{"config": {"base": {"student_model_name": …}}} is a 400. base is required and has no default, so a config naming only one field under it is not a config.
A config carrying a valid base but omitting synthgen or tuning is accepted, and those sections silently take defaults. Dropping one field from a section you do send reverts that field the same way. Nothing errors.

An expand that ran with an override writes the overridden files into the dataset it produces, so config_of("datasets", child_id) reads them back. See How the platform works.

GET /<collection>/<id>/download-metadata returns presigned URLs for an entity’s config.yaml and job_description.json at no credit cost. For a dataset an expand produced it fills only once the job has finished - while it runs, both are null. The config_of helper raises a message naming the entity and its status rather than passing a None to requests.get, which otherwise fails with Invalid URL 'None': No scheme supplied and sends you hunting for a bug in your URL building. A traces object has no config or job description.

GET /slms/<id>/download-metadata returns a third field, model_client_url. Every other entity returns exactly two.

Credits

balances = get("/credits/endpoints")["balances"]
print(balances["datasets_from_datasets_generate_synthetic_data_post"])

The read is free and answers at zero balance. Routes the platform never charges for are absent from the response, so read an unfamiliar key with .get() rather than assuming a zero. A submission against an exhausted route fails 402. POST /datasets/from-datasets is metered by the operation in the body: datasets_from_datasets_relabel_traces_post for the two relabeling operations, datasets_from_datasets_generate_synthetic_data_post for the two generation ones.

The route table and starting balances: How the platform works.

Reading results

teacher = get(f"/teacher-evaluations/{teacher_evaluation_id}/metrics")["teacher_performance"]

slm_metrics = get(f"/slms/{slm_id}/metrics")
base, tuned = slm_metrics["base_model_performance"], slm_metrics["tuned_model_performance"]

Each metrics response also carries a presigned *_download_url for the per-example predictions. Those URLs expire after an hour. The production model’s score is the teacher_performance of a teacher evaluation run with it as the teacher (Teacher evaluation).

Reading a dataset:

download = get(f"/datasets/{dataset_id}/download")
for field, name in [("train_data_url", "train.jsonl"), ("test_data_url", "test.jsonl"), ("traces_url", "traces.jsonl")]:
    if download[field] is not None:
        Path(name).write_bytes(requests.get(download[field]).content)

rows = get(f"/datasets/{dataset_id}/sample")["rows"]     # up to 128 train rows
sizes = get(f"/datasets/{dataset_id}/metrics")            # bytes per file

Downloads are free. A null URL is an empty split. The rows of a split are the parent’s rows followed by the ones the expand added, and nothing in a row says which; download the parent too and diff to isolate the new rows.

What the numbers mean: Metrics.

Uploading a trained model

A model you already have, such as the model.tar and config.yaml downloaded from another SLM, is registered as an SLM by staging the two files. This trains nothing:

body = stage("/staging-slms-s3-urls", {"model": "model.tar", "config": "config.yaml"}, SLM_FIELDS)
slm_id = post("/slms", body)["id"]
poll("slms", slm_id, POLL_TIMEOUT_SECONDS)

The tarball must hold the LoRA adapter under model-adapter/, the layout GET /slms/<id>/download produces (Downloading your model). The config goes beside it, not inside. The platform expands the tarball into the layout a trained model has, which is the job poll waits on, and files that don’t form a valid model answer 400. The route is metered on slms_post, which starts at zero, so ask for a grant before your first upload.

Deployment

A deployment isn’t called directly. Once the job reaches JOB_SUCCESS, an inference endpoint with the deployment as its primary serves it.

A deployment is a session, not a permanent endpoint. It stops after six hours, or after one hour with no traffic, and it can’t be restarted. It bills until that idle timeout, so delete it when you’re finished. Replacing a stopped one spends another deployments_from_slms_post credit, so have your test inputs ready before you create it.

See distil labs inference for querying it.

Inference endpoints

An inference endpoint is an OpenAI-compatible gateway that fronts the model you already run in production and keeps a copy of every call it serves. Those copies are the traces your dataset is built from. Later, a second endpoint serves the trained model with the first model as its fallback. The same thing from the terminal: CLI reference.

endpoint = post("/inference-endpoints", {
    "name-prefix": "support",
    "fallback": {"model": "openai/gpt-4.1-mini"},
    "trace-sample-rate": 1,
})
name = endpoint["unique_endpoint_name"]     # eg. "support-yeOdAS"

name-prefix is appended with a randomized suffix and returned as unique_endpoint_name, which is what every other route takes and what a request body carries.

fallback.model is an OpenRouter model slug in owner/model form, named from the model directory.

trace-sample-rate is the fraction of calls the endpoint records, 0 to 1. Send 1 unless your traffic is very high. An endpoint created without it records one call in a hundred, and the response reports what an endpoint does as trace_sampling_rate.

primary puts a model of your own in front of the fallback, usually a deployment of the trained model, named by its id:

endpoint = post("/inference-endpoints", {
    "name-prefix": "support-slm",
    "fallback": {"model": "openai/gpt-4.1-mini"},
    "primary": deployment_id,
    "trace-sample-rate": 1,
})

The deployment has to have reached JOB_SUCCESS, and the route answers 409 until it has.

For a server you run yourself, primary is an object instead:

"primary": {"url": "https://<your-server>", "api-key": "<your-server-key>"},

primary.url is the base URL of an OpenAI-compatible server, without /v1 since the endpoint appends /v1/chat/completions itself, and primary.api-key the key that authenticates it; both or neither. The endpoint calls the primary first and falls back whenever the primary fails, a timeout included. primary.readiness-gate-timeout-ms is optional and for internal use. An endpoint can’t be changed after creation. See Serving behind the endpoint.

endpoints = get("/inference-endpoints")             # newest first
endpoint = get(f"/inference-endpoints/{name}")      # one, by unique name

POST /inference-endpoints is metered (inference_endpoints_post) and answers 402 once the balance for it is spent, like the other metered routes above.

API keys

Before you can use your endpoint you need to create an API key for it and attach it to your inference endpoint:

key = post("/api-keys", {"api-key-name": "support-prod"})
secret = key["secret"]      # returned by this response and by nothing else

Store secret before moving on as it can not be retrieved again. GET /api-keys lists ids, names and creation dates but never the secret itself. You can attach several API keys to a single inference endpoint so if you lose your key, simply create a new one and delete the old one. Note that the total number of API keys you can have in your account is limited to a maximum of 5.

A key authenticates nothing until it is linked. PUT links it, DELETE on the same path unlinks it, and both answer 204. DELETE /api-keys/<name> revokes the key everywhere it was linked.

requests.delete(
    f"{PLATFORM_URL}/inference-endpoints/{name}/api-keys/support-prod", headers=auth()
)
requests.delete(f"{PLATFORM_URL}/api-keys/support-prod", headers=auth())

Key changes take up to a minute to propagate. A 409 from either link route means the endpoint’s datastore has not caught up with a write yet, usually a create moments earlier, and a link that answered 204 can take a moment longer before the endpoint honours the key. Wait and retry rather than treating either as a failure, and give a new key that minute before you move traffic onto it or revoke the key it replaces.

Calling the endpoint

Our OpenAI compatible inference endpoints are hosted in its own distillabs.ai subdomain and can be called as follows:

requests.post(
    "https://inference.distillabs.ai/v1/chat/completions",
    headers={"Authorization": f"Bearer {secret}", "Content-Type": "application/json"},
    json={"model": name, "messages": [{"role": "user", "content": "…"}]},
)

The model field carries the unique endpoint name. Everything else is an ordinary chat completions request, so any OpenAI-compatible client works against https://inference.distillabs.ai/v1.

Reading the traces

def traces(endpoint_name, from_start_time, to_start_time):
    """Yield an endpoint's traces in a window, newest first, a page at a time."""
    params = {"from-start-time": from_start_time, "to-start-time": to_start_time}
    while True:
        response = requests.get(
            f"{PLATFORM_URL}/inference-endpoints/{endpoint_name}/traces",
            params=params,
            headers=auth(),
        )
        raise_with_body(response)
        page = response.json()
        yield from page["traces"]
        if page["pagination_cursor"] is None:
            return
        params["pagination-cursor"] = page["pagination_cursor"]


records = traces(name, "2026-09-01T00:00:00Z", "2026-10-01T00:00:00Z")

Terminate on pagination_cursor being null or when you’ve accumulated the number of traces you need. The cursor is opaque, so pass it back as given rather than parsing it.

from-start-time and to-start-time set the window as ISO 8601 timestamps. Always send both.

Each trace is the platform’s record of one call. input and output hold the request and the response as JSON strings, and metadata carries the HTTP status and source, which names whether the fallback or the primary answered. Endpoint records to traces has the full shape and the conversion into a trace file.

Exporting the traces to a file

A traces export writes the endpoint’s records in a window to one file, rather than pages:

export = post(f"/inference-endpoints/{name}/traces-exports", {
    "from-start-time": "2026-09-01T00:00:00Z",
    "to-start-time": "2026-10-01T00:00:00Z",
})
Path("raw-traces.jsonl").write_bytes(requests.get(export["download_url"]).content)

The create answers 201 with id, created_at, the window it used and a presigned download_url. Send both bounds, as with /traces. GET /inference-endpoints/<name>/traces-exports lists earlier exports, and GET /inference-endpoints/<name>/traces-exports/<export-id> reads one back with the same fields.

Traces from an endpoint

POST /prepared-traces can also read the endpoint’s records itself, with nothing downloaded, converted or staged:

prepared_traces_id = post("/prepared-traces", {"inference_endpoint_name": name})["id"]

Send traces_jsonl or inference_endpoint_name, not both. The dataset you create from it (Creating a dataset) needs trace_processing.observation_format: langfuse in its config, since the records aren’t {"messages": [...]} lines. Which records it includes and when: Endpoint records to traces.