Prepare Your Training Data

Update November 2, 2026 — Fine-tuning availability. OpenAI’s self-service fine-tuning is being phased out: since May 7, 2026, organizations that have never used it can no longer create jobs; since July 2, you must have used a fine-tuned model within the last 60 days; and starting January 6, 2027, no customer will be able to launch new jobs. Already trained models remain usable until their base model is retired. This course remains useful for understanding and leveraging an existing model; for new projects, prioritize prompt engineering, RAG, and evaluations.

The Importance of Data

The quality of your fine-tuned model depends directly on the quality of your training data. A model trained on poor data will produce poor results, regardless of the number of epochs or hyperparameters chosen: training does not fix a corpus, it amplifies it. This is why this lesson comes before any technical considerations about the API.

We will follow the real path of a dataset: where to find it, how to fill in what’s missing, what volume to aim for, and how to structure each example before formatting it as JSONL in the next lesson.

Data Sources

The first source to explore is your own history, and it is almost always underestimated. If you already run a chatbot or assistant, your conversation logs contain real interactions, with your users’ vocabulary and awkward phrasing—material far more useful than questions invented in a meeting. Your support tickets offer the same richness, with the added bonus of validated resolutions at the end of the exchange. Your internal documents—style guides, procedures, response templates—give you the expected form of answers, if not the questions. Finally, your human evaluations, meaning responses your teams have validated or corrected, are the most valuable examples of the bunch, since they already carry a quality judgment.

When these sources aren’t enough, you can generate synthetic data to cover missing cases:

from openai import OpenAI

client = OpenAI()

def generate_example(topic: str, style: str) -> dict:
    """Generates a synthetic training example."""
    response = client.responses.create(
        model="gpt-5.6-sol",
        input=f"""Generate a realistic conversation between a user
        and an assistant specialized in {topic}.
        The assistant must respond in a {style} style.
        Format: one user message, then the ideal response."""
    )
    return response

Warning: Synthetic data must be verified by a human. Do not use it blindly—a generator that gets a business nuance wrong will produce fifty false examples with the same confidence as fifty correct ones.

How Much Data Do You Need?

OpenAI recommends a minimum of 10 examples, but in practice, 50 is the absolute minimum to observe a difference. Between 100 and 500 examples, you will get reliable results. Beyond that, the gain comes less from volume than from diversity: a thousand reformulations of the same question teach the model nothing more than fifty well-chosen ones.

Structuring Your Examples

Each example must represent a complete interaction, meaning everything the model will see in production plus the response you expect. The system message is optional: it defines the assistant’s role and constraints, and if you decide to use it, it must appear in all your examples. The user message contains the request as it actually arrives, with its imperfections. The assistant message, finally, is the ideal response—not an acceptable one, but the one you would be happy to see served to a customer.

example = {
    "messages": [
        {
            "role": "system",
            "content": "You are a legal assistant specialized in French labor law."
        },
        {
            "role": "user",
            "content": "What is the notice period for a permanent contract?"
        },
        {
            "role": "assistant",
            "content": "The notice period for a permanent contract depends on your seniority and your collective bargaining agreement. As a general rule: less than 6 months seniority, the notice period is set by the collective agreement; between 6 months and 2 years, it is 1 month; beyond 2 years, it is 2 months. Always check your collective agreement, which may provide for different durations."
        }
    ]
}

Preparation Best Practices

Diversity requires deliberate effort. Cover all the use cases you are targeting, include edge cases and ambiguous questions, vary the length of questions and answers, and represent different levels of difficulty. A corpus composed only of easy questions produces a model that stammers at the first twisted request.

Consistency is the inverse constraint, and just as important. All responses must follow the same style and format; if you use a system message, use it in every example; and responses must be what you actually expect in production, not polished versions created for the occasion. The model will learn the average of what you show it, hesitations included.

Finally, there is the train/validation split. Setting aside 20% of examples before training is what will allow you, later, to know whether your model has learned or merely memorized. Do it once, save the two files, and do not mix them again.

import json
import random

# Load your data
with open("complete_data.jsonl", "r") as f:
    examples = [json.loads(line) for line in f]

# Shuffle
random.shuffle(examples)

# Split 80/20
split = int(len(examples) * 0.8)
train = examples[:split]
validation = examples[split:]

# Save
for name, data in [("train.jsonl", train), ("validation.jsonl", validation)]:
    with open(name, "w") as f:
        for ex in data:
            f.write(json.dumps(ex, ensure_ascii=False) + "\n")

print(f"Train: {len(train)} examples, Validation: {len(validation)} examples")

Key Takeaways

• Quality over quantity—100 excellent examples are better than 1,000 mediocre ones
• Manually verify your synthetic data
• Always set aside a validation set (20%) to evaluate objectively
• Ensure consistency of style and format across all your examples