When and Why to Fine-Tune a Model

Update November 2, 2026 — Fine-tuning availability. OpenAI’s self-service fine-tuning is being phased out: since May 7, 2026, organizations that had never used it can no longer create jobs; since July 2, you must have used a fine-tuned model within the last 60 days; and starting January 6, 2027, no customer will be able to launch new jobs. Already trained models remain usable until their base model is retired. This course remains useful for understanding and leveraging an existing model; for new projects, prioritize prompt engineering, RAG, and evaluations.

What Is Fine-Tuning?

Fine-tuning involves retraining an existing language model on your own data to adapt its behavior to your specific needs. Where prompt engineering guides the model with contextual instructions—rewritten, sent, and billed on every call—fine-tuning modifies the model’s internal weights to permanently embed new behaviors. Once training is complete, what you previously described in a long system prompt becomes part of the model itself.

At OpenAI, fine-tunable models follow the current catalog (the GPT-5.6 family since July 2026)—check the official fine-tuning documentation for the exact list of variants open to training. The workflow itself doesn’t change: you create a training dataset in JSONL format, launch a job via the API, and receive a custom model that you call exactly like the base model. In your code, only the model identifier changes.

When Is Fine-Tuning Appropriate?

Fine-tuning isn’t always the right answer, and that’s the first thing to internalize before writing a single line of JSONL. It delivers real value primarily for tone and style: if your product requires a specific voice, training it on a few hundred exchanges already written in that voice saves you from repeating those instructions in every prompt. The second strong use case is structured output formatting. When your pipeline expects JSON with a schema that must never vary, well-written instructions get you close but never guarantee 100% compliance; systematically compliant examples reduce that gap to nearly zero.

These two drivers mechanically lead to two others. A fine-tuned model queried with a short prompt responds faster than a base model burdened with a long system prompt, which reduces latency perceived by the user; and fewer input tokens mean a lighter bill at constant volume, resulting in cost reduction that shows up directly on a service handling several thousand requests per day.

Two additional task families stand out where fine-tuning excels. The first covers classification tasks: categorizing support tickets, analyzing sentiment, sorting documents. The expected output here is short, closed-ended, and perfectly illustrated by example. The second is consistent function calling: when the model must systematically call the right functions with the right parameters, showing it a hundred correct calls works better than describing the rule.

Conversely, four situations should steer you away. If you have little data, below 50 examples, prompt engineering with a few few-shot examples will perform better for far less effort. If your domain changes quickly, with a product catalog or pricing that shifts weekly, RAG (Retrieval-Augmented Generation) will save you from constantly retraining. The most common confusion involves trying to inject factual knowledge: fine-tuning teaches behaviors, not facts, and here again RAG is the right tool. Finally, if prompt engineering suffices, if a few well-crafted instructions already yield good results, don’t complicate your pipeline for the sake of it.

Fine-Tuning vs. Alternatives

These three approaches aren’t competitors: they address different needs and combine very well, for example a fine-tuned model for formatting fed by RAG for facts.

Typical Workflow

The rest of this course follows the order in which you’ll actually execute these steps. Take a few minutes before continuing to write down the first one: articulating the desired behavior in a single sentence will save you from collecting data at random.

1. Identify the need: what behavior do you want the model to adopt systematically?
2. Collect data: gather at least 50 to 100 examples of ideal conversations
3. Prepare the JSONL: format your data according to the schema expected by the API
4. Launch training: create a fine-tuning job via client.fine_tuning.jobs.create()
5. Evaluate: compare the fine-tuned model’s performance against the base model
6. Iterate: adjust your data and rerun if necessary

Key Takeaways

• Fine-tuning changes the model’s behavior, not its factual knowledge
• It makes sense when you have sufficient data and a need for consistency
• Always start with prompt engineering—only fine-tune if that’s not enough
• The list of fine-tunable models evolves with OpenAI’s catalog—check the official documentation before launching a job