Synthetic data generation

Your seed dataset has a few dozen examples. Training needs thousands. This stage is where the teacher writes them: it generates examples from your seed data, job description and mutators, then validates and de-duplicates them into a training dataset.

The result is a merged dataset. The root train.jsonl is your seed examples plus the surviving synthetic ones, so the synthetic count is the total minus your seed count.

This stage also runs from an empty seed set. It runs zero-shot from your task description, your tool or class definitions and unstructured.jsonl, and what it generates becomes the training set. No seed rows set the format or style in that case, so read the output closely.

Running it

A smoke run is the same job with generation_target set to 128 and the rest of your config unchanged, so you can check the data before generating all of it. Start a smoke run on your seed dataset. The command prints the new training dataset’s id:

distil training-dataset create-from-seed-dataset --smoke --output json <seed-dataset-id> | jq -r .id

Check its status with that id until it reads JOB_SUCCESS:

distil training-dataset status --output json <training-dataset-id> | jq -r .status

The full job is the same command without --smoke. The default generation_target is 10000; set another size if you need one. A full run of that size takes about 90 minutes. Generation runs in batches of generation_iteration_size, so the result can land a little above generation_target.

A smoke spends a training_datasets_from_seed_datasets_smoke_post credit, and a full run a training_datasets_from_seed_datasets_post credit. Both take --config and --job-description overrides on the same seed dataset (How the platform works).

Check your training credits before generating

A generation run with no training credit left produces data you can’t train on. Run distil credits-balance and read it against your whole plan.

What controls the generated data

task_description defines what a correct answer is. It also feeds evaluation and the judge, so it stays fixed and matched to your production prompt. Three things control the data:

• synthetic_data_generation_instructions in job_description.json: a constant added to every generation call that describes the inputs to write (formats, domains, register, noise). It changes every generated row the same way, so use it when the whole training set is off in the same direction.
• Mutators in synthgen.mutators: the dimensions the data is spread over. Each generation call samples one value per mutator, so they set the mix of the data rather than every row. Use them for coverage: slices that are missing, rare, or over-represented against production.
• The synthgen config: how much is generated, how much survives validation and how much each call writes. These change the volume and cleanliness of the data, not what it is about.

Mutators

synthgen.mutators holds one entry per dimension you want to vary:

synthgen:
  mutators:
    - name: complexity
      values: [simple, medium, complex]
    - name: topic
      values:
        - "about billing disputes: refunds, double charges, invoice errors"
        - "about account cancellation: downgrades and win-back attempts"
        - "about technical support: setup problems and integration errors"

None are applied by default: with no mutators, every example comes from the same prompt.

Field	Required	Notes
name	Yes	The dimension. Unique across mutators
values	No	The values, each a plain string. Leave it out to have the teacher detect them, which requires a description
method	No	fixed (default) samples from the target distribution. adaptive measures what survived validation and samples more of the values that fall behind
target_distribution	No	uniform (default), match_seed (the proportions the teacher finds in your seed data), or one weight per value, such as [5, 3, 2]
description	No	What the dimension means. Required when values is left out; read by the classifier with adaptive and match_seed
auto_detect_sample_size	No	Seed examples the teacher reads when detecting values. Default 50
beta	No	adaptive only, 0 to 1, default 0.9. How far sampling may move away from the target

A value is one string. Put any detail after a colon and quote the whole value, so YAML reads a string: - "simple: minimal reasoning required" is a value, while - simple: minimal reasoning required is a mapping and is rejected.

Each mutator renders one line in the generation prompt. With the first example above, one call’s prompt can contain:

Important! Make sure to follow those guidelines:
Generated examples should be simple
Generated examples should be about billing disputes: refunds, double charges, invoice errors

Every line is Generated examples should be followed by the value, so phrase values to finish that sentence (about billing disputes, not billing).

Use one mutator per independent dimension, 1-2 mutators of 3-10 values each. To skew the mix, give one weight per value: values: [English, French] with target_distribution: [3, 1] asks for English in about three calls out of four.

Letting the teacher pick the values. If you know the dimension but not its values, leave values out and describe the dimension:

synthgen:
  mutators:
    - name: topic
      description: which part of the product the customer is asking about

The teacher proposes the values once, at the start of the run, from your job description and a sample of your seed data. The values are printed in the run log; copy them into values to edit them or to get the same values on every run.

Adaptive mutators. A fixed mutator never checks what came back: the teacher can ignore a directive, and validation can drop one value’s examples more than another’s. With method: adaptive, the teacher labels the surviving examples every synthgen.mutator_update_frequency batches (default 5), and sampling moves toward the values that fall behind. Each labeled example is one teacher call, so this makes generation slower. Always write the description for an adaptive mutator.

Tools. For chat completion with tools, add a mutator with one value per tool, so every tool is covered (Chat completion).

In classification, a mutator whose values bear on the label, such as a severity or a category, can lead the teacher to the wrong class. Mutators that leave the label to classes_description (style, length, phrasing) avoid this; check the labels of the generated rows when mutators are on.

Configuring generation

The full table is in Config file. The ones to set deliberately:

• validation_max_total_length. The character cap on one example (question, answer and context together). Set it to the longest combined length you expect: the default of 30,000 rejects longer uploaded examples and filters out longer generated ones.
• generation_in_single_call and the exemplar counts. Lower them when examples are long; each multiplies the prompt and output size of a teacher call. No exemplar count may exceed the number of training rows.
• output_is_json: true. When answers must be valid JSON. Question answering only.
• base.enable_thinking. For a reasoning student, set here; see Reasoning models.

Checking what came back

Save a sample of the training data to sample.json, which is free, and read the size of the training data:

distil training-dataset sample --output json <training-dataset-id> > sample.json
distil training-dataset metrics --output json <training-dataset-id> | jq .train_data_size_bytes

The sample holds at most 128 rows drawn from the training data, and the training data holds your seed rows and the generated rows together with nothing marking which is which. Compare the sample against your seed rows to tell them apart. The sample can hold few or none of the generated rows, notably when the seed is large; when it doesn’t cover them, download the full dataset instead. Metrics report bytes, not rows. Downloading the whole dataset costs a training_datasets_download_get credit, which new accounts don’t have.

Compare the count against your target first. Falling well short means validation filtered heavily, usually on length or format. Then check three things on the generated rows:

1. Form. Each row parses, carries the required output format and respects the stated constraints. A malformed row is a defect whatever the rate. A handful of answers shows whether the teacher understood the task; a low rate of wrong labels from the teacher is tolerated by the student.
2. Distribution. Compare generated against seed data along the dimensions that matter for your task: length, topic coverage, style and register, class balance.
3. Requested slices. Count each slice your mutators or job description asked for: a mutator value is a request and can produce nothing. A smoke makes about 32 mutator draws at the default generation_in_single_call of 4, so check this on the smoke only when the mutator grid has at most 16 cells; with more, check it on the full run.

If the dataset isn’t good

Problems in the training data show up in training only as a score that stops improving, after a full training run. The instructions, the mutators and the config are what change the data, and a new dataset costs one generation credit, while a training run on a bad one costs a training credit and the full training time.

Scattered label errors are noise the student tolerates. When the teacher gets one slice wrong again and again, for example mislabels one class or breaks a rule or the output format, there are two fixes:

• Describe that slice and its correct handling in more detail in synthetic_data_generation_instructions, or use a stronger teacher (Supported models), and generate again.
• Download the training data, correct the rows, and upload the directory as a new training dataset with distil training-dataset create --data <dir>. This spends a training_datasets_download_get and a training_datasets_post credit.

In the next iteration

The last training dataset can be the seed of the next run: download it, create a seed dataset from the directory with new mutators, and set generation_target to the number of rows to add. The result is the old rows plus the new ones (Model iterations).

Next

Model training.