Endpoint records to traces

The endpoint’s records become the traces of your dataset, the input of Building the test set. Write the dataset’s config.yaml and job_description.json first (Dataset overview); then take one of two routes.

Directly from the endpoint

The platform reads the endpoint’s records itself. A call becomes available about 24 hours after it was made, and every record the endpoint holds goes in. Create the traces object, then the dataset from it:

distil traces create-from-inference-endpoint support-yeOdAS
distil dataset create-from-traces --config config.yaml \
  --job-description job_description.json <traces-id>

The first command prints Prepared traces created. ID: <traces-id>, the second Dataset created. ID: <dataset-id>. No job runs: the traces are copied into the dataset, and its train and test splits are empty. The config has to name the format of the endpoint’s records:

trace_processing:
  observation_format: langfuse

Download, convert, upload

Take this route when you need the traces now, or when the records must be filtered first, for example to keep only one model’s answers once a trained model serves the endpoint. Download the records:

distil inference-endpoint download-traces --from-start-time 2026-09-01 --all support-yeOdAS

Each line is one call: the request under input, the response under output, and metadata with the HTTP status and the source (fallback or primary). Convert the successful calls into conversations, one {"messages": [...]} per line. An agent makes one call per step; for the chat completion tasks the platform drops a trace whose messages are a prefix of another trace’s, so each loop counts once.

import json


def convert(record):
    if record["metadata"].get("status") != 200:
        return None
    request = json.loads(record["input"])
    reply = json.loads(record["output"])["choices"][0]["message"]
    if not reply.get("content") and not reply.get("tool_calls"):
        return None
    assistant = {"role": "assistant", "content": reply.get("content") or ""}
    if reply.get("tool_calls"):
        assistant["tool_calls"] = reply["tool_calls"]
    converted = {"messages": [*request["messages"], assistant]}
    if request.get("tools"):
        converted["tools"] = request["tools"]
    return converted


with open("support-yeOdAS-traces.jsonl") as src, open("input-dir/traces.jsonl", "w") as dst:
    for line in src:
        converted = convert(json.loads(line))
        if converted is not None:
            dst.write(json.dumps(converted, ensure_ascii=False) + "\n")

Put the result next to config.yaml and job_description.json, with observation_format: openai_messages (the default) in the config, and create the dataset:

distil dataset create --data ./input-dir

The command prints Upload successful. Dataset ID: <dataset-id>.

Next

Build the test set.