Costs and Optimization

Update November 2, 2026 — Fine-tuning availability. OpenAI’s self-service fine-tuning is being phased out: since May 7, 2026, organizations that have never used it can no longer create jobs; since July 2, you must have used a fine-tuned model within the last 60 days; and starting January 6, 2027, no customer will be able to launch new jobs. Already trained models remain usable until their base model is retired. This course remains useful for understanding and leveraging an existing model; for new projects, prioritize prompt engineering, RAG, and evaluations.

Understanding the Cost Structure

OpenAI fine-tuning involves two types of costs: a one-time training cost and a recurring inference cost. Understanding and optimizing these costs is essential for a profitable deployment. The distinction isn’t just accounting: the first is a one-time decision made before launching the job, and it doesn’t change; the second follows you every month and grows with your traffic. A team that haggles over training costs while letting a thousand-token prompt run on five thousand daily requests is fighting the wrong battle.

The Training Cost

Training is billed per token processed, and that number depends directly on two variables you control: the size of your dataset and the number of passes over it.

Training cost = total_tokens × number_epochs × price_per_training_token

This formula lets you estimate the bill before launching anything. The following function scans the JSONL file and approximates the token count based on message length—the estimate of one token per four characters in French is rough, but it’s enough to distinguish a few-dollar job from one costing several hundred. Make it a habit to run this systematically: a preparation error that triples your dataset shows up immediately in the result.

import json

def estimate_training_cost(
    jsonl_file: str,
    n_epochs: int,
    price_per_million_tokens: float
) -> dict:
    """Estimates training cost before launching the job."""
    total_tokens = 0

    with open(jsonl_file, "r") as f:
        num_examples = 0
        for line in f:
            num_examples += 1
            obj = json.loads(line)
            for msg in obj["messages"]:
                # Rough estimate: 1 token ~ 4 characters in French
                total_tokens += len(msg["content"]) // 4

    training_tokens = total_tokens * n_epochs
    cost = training_tokens * price_per_million_tokens / 1_000_000

    return {
        "examples": num_examples,
        "tokens_per_pass": total_tokens,
        "epochs": n_epochs,
        "total_tokens": training_tokens,
        "estimated_cost": f"${cost:.2f}"
    }

result = estimate_training_cost(
    "training_data.jsonl",
    n_epochs=3,
    price_per_million_tokens=8.0  # check current pricing
)

for k, v in result.items():
    print(f"{k}: {v}")

The Inference Cost

Inference is billed per call, based on input and output tokens, and fine-tuned models generally have a slightly higher rate than the base model. This unit surcharge often causes hesitation, mistakenly: it applies to far fewer tokens, since the behavior previously described in a long system prompt is now built into the model. The calculation below puts both scenarios side by side for a month of identical traffic, with a prompt dropping from 800 to 100 tokens against doubled rates. Adjust the parameters with your own numbers before drawing conclusions: for usage where the prompt was already short, the trade-off reverses.

def monthly_cost(
    requests_per_day: int,
    avg_input_tokens: int,
    avg_output_tokens: int,
    price_input_per_million: float,
    price_output_per_million: float
) -> float:
    """Calculates monthly inference cost."""
    requests_month = requests_per_day * 30
    input_cost = requests_month * avg_input_tokens * price_input_per_million / 1_000_000
    output_cost = requests_month * avg_output_tokens * price_output_per_million / 1_000_000
    return input_cost + output_cost

# Scenario: base model with long prompt
base_cost = monthly_cost(
    requests_per_day=5000,
    avg_input_tokens=800,    # long system prompt + question
    avg_output_tokens=200,
    price_input_per_million=0.15,
    price_output_per_million=0.60
)

# Scenario: fine-tuned model with minimal prompt
ft_cost = monthly_cost(
    requests_per_day=5000,
    avg_input_tokens=100,    # short prompt, behavior integrated
    avg_output_tokens=200,
    price_input_per_million=0.30,  # higher FT rate
    price_output_per_million=1.20
)

print(f"Monthly base cost: ${base_cost:.2f}")
print(f"Monthly fine-tuned cost: ${ft_cost:.2f}")
print(f"Difference: ${ft_cost - base_cost:+.2f}")

Reducing Input Tokens

The main lever for savings with a fine-tuned model is prompt reduction. Every instruction you manage to remove because the model has internalized it is a gain repeated on every call, until the service reaches end of life. The table below gives the observed orders of magnitude depending on the degree of integration achieved.

Don’t remove these instructions all at once hoping everything will hold. Remove them in groups, re-running your test set after each removal: you’ll discover that some rules have been perfectly learned while others, less represented in your examples, must stay in the prompt.

Balancing Dataset Size and Epochs

More data gives a better model, but also a higher training cost, and returns diminish. Going from one hundred to three hundred examples usually changes a lot; going from one thousand to three thousand changes little, for three times the price. Train several sizes, measure the quality achieved, and relate it to cost to find your break-even point.

def find_optimal(results_by_size: dict):
    """Identifies the optimal dataset size."""
    print("Dataset size | Quality score | Training cost")
    print("-" * 55)

    best_ratio = 0
    optimal_size = 0

    for size, data in sorted(results_by_size.items()):
        score = data["score"]
        cost = data["cost"]
        ratio = score / cost if cost > 0 else 0

        print(f"{size:>14} | {score:>13.2f} | {cost:>17.2f} $")

        if ratio > best_ratio:
            best_ratio = ratio
            optimal_size = size

    print(f"\nOptimal size: {optimal_size} examples")

The choice of base model weighs on the same scale. Luna is cheaper than Terra, both for training and inference: always start with the cheapest model and move up only if the quality achieved is insufficient. Check which models are eligible for fine-tuning in the OpenAI documentation—the list changes with each generation and never covers the entire catalog. Then there’s the number of epochs, since each additional pass mechanically multiplies the training cost. If 2 epochs give 95% of the quality of 4 epochs, the choice is quick.

Monitoring Spending in Production

The estimate made before launch applies only to training. In production, it’s actual traffic that drives costs, and it often surprises on the upside. A counter fed on every call, tied to a monthly budget, alerts you before the bill does. The 80% threshold leaves time to react: reduce a prompt, cap response lengths, or shift some traffic to a cheaper model.

class CostTracker:
    """Real-time cost tracking."""

    def __init__(self, monthly_budget: float):
        self.budget = monthly_budget
        self.spending = 0.0
        self.calls = 0

    def record(self, input_tokens: int, output_tokens: int,
               input_price: float, output_price: float):
        """Records the cost of a call."""
        cost = (
            input_tokens * input_price / 1_000_000 +
            output_tokens * output_price / 1_000_000
        )
        self.spending += cost
        self.calls += 1

        # Alert if approaching budget
        utilization = self.spending / self.budget * 100
        if utilization > 80:
            print(f"ALERT: {utilization:.1f}% of monthly budget used")

    def report(self):
        """Consumption report."""
        print(f"Calls: {self.calls}")
        print(f"Spending: ${self.spending:.2f}")
        print(f"Remaining budget: ${self.budget - self.spending:.2f}")
        if self.calls > 0:
            print(f"Average cost per call: ${self.spending / self.calls:.4f}")

Justifying the Investment

There remains the question your leadership will ask sooner or later: how long until the training pays for itself? The calculation relates a one-time cost to a monthly benefit, which adds up token savings and the value you assign to quality gains. This second term is the trickiest, and it’s better to leave it at zero than to invent it: a payback demonstrated on measurable savings alone carries more conviction than a generous projection.

def calculate_roi(
    training_cost: float,
    monthly_savings: float,
    quality_gain_value: float = 0
) -> dict:
    """Calculates the return on investment for fine-tuning."""
    monthly_benefit = monthly_savings + quality_gain_value
    payback_months = training_cost / monthly_benefit if monthly_benefit > 0 else float("inf")

    return {
        "training_cost": f"${training_cost:.2f}",
        "monthly_benefit": f"${monthly_benefit:.2f}",
        "payback": f"{payback_months:.1f} months",
        "annual_roi": f"{(monthly_benefit * 12 - training_cost) / training_cost * 100:.0f}%"
    }

Key Takeaways

• Estimate training cost before launching the job
• The main lever for savings is reducing input tokens
• Start with the cheapest base model (Luna) before moving up to Terra
• Monitor costs in production with budget alerts
• Calculate ROI including quality gains and token savings