How the platform works

This page describes how the platform behaves, whether you drive it with the CLI yourself, through the agent skill or over the REST API, since those only differ in how you phrase a request. The commands are in the CLI reference and the requests in Using the REST API.

Entities and jobs

Four entities carry the data, and each is created either from files you supply or by running a job over the entity before it:

traces object ─► dataset ─► dataset ─► dataset … ─┬─► teacher evaluation
(traces.jsonl)   (traces,   (one expand            └─► SLM ─► deployment
                 no rows)    operation each)

• A traces object holds one traces.jsonl and nothing else. It is uploaded, or created from an inference endpoint’s records. It runs no job.
• A dataset holds config.yaml, job_description.json, train.jsonl, test.jsonl and traces.jsonl; any data file can be empty or missing. It is created in three ways, recorded in its operation:
  • direct_upload: the files on disk, any subset of the three data files. No job runs.
  • from_prepared_traces: a traces object plus a config and a job description. The traces are copied, train and test are empty. No job runs.
  • an expand operation on a parent dataset, one of relabel_traces_train, relabel_traces_test, generate_synthetic_data_train and generate_synthetic_data_test. A job runs and writes a complete new dataset; the parent is unchanged.
• A teacher evaluation and an SLM are created from any dataset; the SLM is trained on its train.jsonl and scored on its test.jsonl.

A dataset has at most one parent (parent_prepared_traces_id or parent_dataset_id), so a project is a chain of datasets, each one expand away from the previous. Keep each ID as you go; distil dataset show <id> prints a dataset’s parent, and distil dataset list every dataset with its operation.

An uploaded traces object, an uploaded dataset and a dataset created from traces run no job: they exist only once the create has succeeded, their status is JOB_SUCCESS as soon as the create returns, and their logs are empty. A dataset made by an expand has a job behind it, and its status says how the job is going.

The expand operations

Each operation reads one dataset and writes a new one with rows added to one split:

Operation	Reads from the parent	What the new dataset adds
relabel_traces_test	trace_processing.num_test_relabelled traces	relabeled rows in test.jsonl
relabel_traces_train	trace_processing.num_train_relabelled traces	relabeled rows in train.jsonl
generate_synthetic_data_test	the test.jsonl rows as examples, traces as context	synthgen.test_generation_target synthetic rows in test.jsonl
generate_synthetic_data_train	the train.jsonl rows as examples, traces as context	synthgen.train_generation_target synthetic rows in train.jsonl

Four rules apply to all of them:

• Each expand removes the traces it used. A trace is used at most once along a chain, so test and train never come from the same trace, and the trace pool shrinks with every expand.
• Relabeling needs the traces it is asked for. The job fails when the dataset holds fewer distinct traces than num_{split}_relabelled. Fewer rows than traces can come out, because filtering and schema checks drop some; zero rows is still a success.
• Generation uses traces as context when there are enough. With synthgen.use_traces_as_context: true (the default) and a target of T rows, the job takes T + min(T, 1000) traces as context, or all that are left, and removes them. With fewer than min(T / 4, 10) traces it logs a warning, generates without context and leaves the traces alone. The split’s existing rows are the examples; an empty split generates zero-shot.
• The result is validated before it is written. A new dataset that breaks a dataset rule, for example a row that is longer than synthgen.validation_max_total_length, fails the expand.

Generation also drops new rows that duplicate rows already in the split (synthgen.validation_similarity_threshold) or any row of the other split. The other split and the config and job description are carried over with the same content. Relabeling writes traces.jsonl back in openai_messages form without the system prompts, unless base.variable_system_prompt is on. Nothing in a row says whether it was uploaded, relabeled or generated: to tell the new rows apart, compare against the parent dataset, whose download is free.

Job status

Status	Meaning	Terminal
JOB_NOT_STARTED	accepted, not yet scheduled	no
JOB_PENDING	scheduled, waiting for compute resources to be provisioned; logs are empty	no
JOB_RUNNING	running	no
JOB_SUCCESS	finished; outputs readable	yes
JOB_FAILURE	failed	yes
JOB_STOPPED	stopped deliberately	yes

There is no JOB_FAILED. For training, JOB_RUNNING covers the training and the evaluation after it, and the metrics appear only at JOB_SUCCESS.

Deployments report deployment_status and endpoint_status instead of status.

Creating a job returns its ID at once, and only the status says how the job is going. How long a job takes depends on the size of the data and the problem. Submit, then poll between other work until the status is terminal.

Reading a failed job

When a job fails, its log holds the cause, but not always at the end. A crash often ends with a second, unrelated error, so an out-of-memory failure can finish with a pickling error thousands of characters after the OutOfMemoryError that caused it. Search the log for the first error rather than reading the last one.

A failed job produces no output: no dataset rows, no metrics. Fix the cause and submit again: the retry is a new entity from the same parent. Config and job description are readable on some failed entities and not others, so keep a copy of what you submitted.

Smoke runs

The four expand operations and training take a --smoke flag that runs the job small, on the same parent and otherwise with the same config:

• Relabel traces smoke: the split’s num_{split}_relabelled is forced to 128.
• Synthetic data generation smoke: the split’s generation target is forced to 128.
• Training smoke: one epoch on the 128 longest train rows and the 32 longest test rows of the dataset. The rows are chosen by length alone, not per class.

All accept config and job-description overrides, and all spend the smoke credit routes, never a full-run credit. A smoke expand writes a dataset of its own, a side branch of the parent: the full run is submitted on the same parent, not on the smoke’s child, so the smoke never costs the full run any traces.

Never continue from a smoke child: its config carries the smoke values. The full run, and every expand after it, is submitted on the smoke’s parent.

Overrides

A job’s config.yaml and job_description.json can each be varied at submission. An iteration is the same parent submitted again rather than a new upload, so several attempts can all point at one parent dataset.

Two rules govern this. Neither raises an error when you get it wrong.

Each file is replaced whole, never merged

Anything you don’t send takes its default from Config file, not the parent’s value, and that applies to individual fields as well as whole sections. Sending only the section you’re changing reverts everything else without an error. So every override starts by reading the parent’s file, editing one field, and sending it back entire.

An override reaches the config and the job description and nothing else. Changing the rows means a new dataset: download the parent, edit the files, upload the directory. An expand that ran with an override writes the overridden files into the new dataset, so every dataset after it inherits the change. See Config file and the CLI reference.

Credits

Credits are metered per route rather than from one pool, so a balance is a count of remaining calls to that route. Reading the balance (distil credits-balance) is free and answers at zero.

What	Route key	New account starts with
Traces object (upload, or from an endpoint)	prepared_traces_post	100
Dataset upload, also the validator	datasets_post	5
Dataset from a traces object	datasets_from_prepared_traces_post	5
Relabel traces, train or test, full	datasets_from_datasets_relabel_traces_post	5
Synthetic data generation, train or test, full	datasets_from_datasets_generate_synthetic_data_post	5
Any expand, smoke	datasets_from_datasets_smoke_post	20
Teacher evaluation	teacher_evaluations_post	5
Model training, full	slms_from_datasets_post	5
Model training, smoke	slms_from_datasets_smoke_post	20
Model upload (distil slm create)	slms_post	0
Deployment	deployments_from_slms_post	2
Inference endpoint	inference_endpoints_post	10

Downloads are free. slms_post starts at zero, so what it gates is unavailable until you ask for a grant at contact@distillabs.ai. Read a balance against your whole plan rather than the next submission: a test set, a train set and a model are at least four full expand credits and one training credit, and a synthetic data run is wasted if no training credit is left after it.

A call against an exhausted route is refused.

Only metered routes appear in a balance. A route the platform never charges for is absent rather than reported as unlimited, so a missing key isn’t a zero.

What each entity produces

Entity	Metrics	Files	Config + job description
Traces object	-	traces.jsonl	- (it holds neither)
Dataset	train, test and traces size in bytes	all five files, free	free
Teacher evaluation	teacher score, its predictions	-	free
SLM	base and tuned scores, the tuned model’s predictions	model tarball (the LoRA adapter)	free; also the inference client
Deployment	-	-	-

Metrics and files answer once that entity’s own job reaches JOB_SUCCESS. Before that a metrics field is null and a download names the job’s state and fails. An empty split is an empty file and reads 0 in the metrics. Config and job description are settled at submission, so a teacher evaluation or SLM answers within seconds of the create; a dataset made by an expand answers only once its job succeeds.

Two conditions apply on top of that table:

• No entity scores your production model on its own. The baseline the student must beat is a teacher evaluation on the dataset with the production model as base.teacher_model_name (Teacher evaluation).
• An SLM reports both scores but only the tuned model’s predictions. The base-versus-tuned gap is available as numbers; a base-model failure case isn’t.

Runs aren’t reproducible

The teacher generates and the judge scores at non-zero temperature, so the same config submitted twice gives different data and different numbers. base.random_seed doesn’t pin this. Treat every score as a sample (Variation between runs).