Best Practices and Recap

Course Recap

You have explored the full range of structured output mechanisms offered by the Mistral API. This final lesson brings together what makes the difference in daily work: how to choose between the two approaches, the non-negotiable settings, the most common pitfalls, and what to monitor once you are in production.

Choosing Your Approach

Four questions are enough to decide, in this order. Do you have a defined output schema? If the answer is no, stick with JSON Mode: you are in exploration or prototyping mode, and locking down a data model would be premature. If yes, move to the next question.

Are you working in Python or TypeScript? If so, Custom Structured Outputs are ready for you with Pydantic or Zod. If not, JSON Mode with manual validation remains your path.

Are you in production? If yes, Custom Structured Outputs are mandatory, always. Otherwise, JSON Mode is more than sufficient for prototyping.

Does the schema change frequently? A still-evolving schema makes JSON Mode more practical at first, until requirements stabilize; a fixed schema argues for Custom Structured Outputs and the reliability they bring.

The Ten Golden Rules

The first five concern how you write your calls. Always set temperature=0 in production: temperature variability is the enemy of reliability, and extraction has no need for creativity.

# Production: deterministic
response = client.chat.parse(
    model="mistral-large-latest",
    messages=messages,
    response_format=MonSchema,
    temperature=0  # Reproducibility
)

Always check finish_reason. This is the only indicator that warns you of a truncated response before you attempt to use it.

if response.choices[0].finish_reason != "stop":
    # Response is truncated — increase max_tokens
    raise ValueError(f"Incomplete response: {response.choices[0].finish_reason}")

In JSON Mode, always instruct the model in the prompt. The parameter guarantees syntax, not structure: it is up to you to state the keys and their types.

# JSON Mode — the prompt MUST describe the structure
messages = [
    {
        "role": "system",
        "content": "Return JSON with keys: name (string), age (number), city (string)."
    },
    {"role": "user", "content": text}
]

In Custom mode, document your fields. A precise description is worth more than a long prompt, and it travels with the schema in both languages.

class Product(BaseModel):
    name: str = Field(description="Commercial product name")
    price: float = Field(description="Price in EUR excluding tax, rounded to 2 decimals")

const ProductSchema = z.object({
  name: z.string().describe("Commercial product name"),
  price: z.number().describe("Price in EUR excluding tax, rounded to 2 decimals"),
});

Separate context from content. Instructions live in the system message, the data to process in the user message—never the reverse, or the model will confuse the instruction with the source text.

messages = [
    {"role": "system", "content": "You are a data extractor. [Format instructions]"},
    {"role": "user", "content": "[Text to process — NO format instructions here]"}
]

The next two rules concern your safety net. Implement a retry mechanism: never make a single call without a backup; retry with feedback is the minimal pattern in production. And validate beyond the schema, since the schema guarantees structure while semantic validation—bounds, consistency, business logic—remains your responsibility.

The last three relate to orchestration. Choose the right model for the right use case: mistral-large-latest for complex tasks and multi-field extraction, mistral-small-latest for simple classification and basic extraction, ministral-8b-latest for high frequency and minimal latency. Log every call in production, recording the model used, duration, finish_reason, and outcome: without logs, you are blind. Finally, do not over-optimize the prompt too early—start simple, measure, iterate; premature prompt optimization is a common trap, whereas the schema and validation are the real levers.

Three Costly Pitfalls

The first is a max_tokens value that is too low. JSON will be truncated on long responses, and the error will only manifest on large documents, often after going live.

# Dangerous — JSON will be truncated for long responses
response = client.chat.parse(
    model="mistral-large-latest",
    messages=messages,
    response_format=MonSchema,
    max_tokens=128  # Too low for a complex schema
)

Set max_tokens based on the expected size of your output: an invoice with 20 lines requires more tokens than a simple classification.

The second is non-zero temperature, which makes your results non-reproducible for zero gain on an extraction task.

# Dangerous in production — non-reproducible results
response = client.chat.parse(
    ...,
    temperature=0.7  # Unnecessary variation for extraction
)

The third is the reflex to use a single model. Do not use mistral-large for everything: a binary classification does not require the most powerful model, and ministral-8b will do the job 5x faster for less cost.

What to Monitor

Four metrics tell the health of your pipeline. The success rate measures the percentage of calls that return a valid object on the first try; the retry rate, the percentage of calls that needed a second chance. P50/P95 latency gives the median response time and the 95th percentile, which is more revealing than the average. Cost per extraction, expressed in tokens consumed per call, keeps the bill under control.

On these metrics, a few alert thresholds hold up well: a success rate below 95% justifies a yellow alert, below 85% a red alert; P95 latency greater than 10 s warrants investigation; and an open circuit breaker calls for an immediate alert.

Going Further

Four directions naturally extend this course. Function calling uses structured outputs to feed function calls in an agent. Streaming allows you to receive the structured output progressively, which matters for long responses. Fine-tuning adjusts a Mistral model to improve extraction quality on your specific data. Evaluation, finally, consists of setting up an automatic framework based on a golden dataset and metrics—this is what will tell you if a prompt change actually improves things.

A Complete Pipeline

The following code brings together the course practices in a single flow: a schema described field by field, a validator that rejects out-of-bounds scores, a model sized to the task, temperature=0, an explicit max_tokens, verification of finish_reason, and logging for both possible outcomes.

from mistralai.client import Mistral
from pydantic import BaseModel, Field, field_validator
import logging
import time

logger = logging.getLogger("pipeline")

class CustomerFeedback(BaseModel):
    sentiment: str = Field(description="positive, negative, or neutral")
    score: float = Field(description="Score between -1.0 and 1.0")
    category: str = Field(description="product, service, delivery, price, or other")
    action_required: bool = Field(description="True if action is required")
    summary: str = Field(description="One-sentence summary")

    @field_validator("score")
    @classmethod
    def score_in_range(cls, v):
        if not -1.0 <= v <= 1.0:
            raise ValueError(f"Score out of bounds: {v}")
        return v

def analyze_feedback(client: Mistral, text: str) -> CustomerFeedback:
    start = time.time()
    try:
        response = client.chat.parse(
            model="mistral-small-latest",
            messages=[
                {"role": "system", "content": "Analyze customer feedback."},
                {"role": "user", "content": text}
            ],
            response_format=CustomerFeedback,
            temperature=0,
            max_tokens=256
        )
        assert response.choices[0].finish_reason == "stop"
        result = response.choices[0].message.parsed
        logger.info(f"OK in {time.time()-start:.2f}s — {result.sentiment} ({result.score:+.1f})")
        return result
    except Exception as e:
        logger.error(f"ERROR in {time.time()-start:.2f}s — {e}")
        raise

About thirty lines, and you have a component that can be plugged safely into the middle of a system: this is exactly what structured outputs make possible.

Key Takeaways

• Use Custom Structured Outputs in production, JSON Mode for prototyping
• temperature=0, checking finish_reason, and retry are non-negotiable
• Choose the model based on task complexity and budget
• Monitor success rate, latency, and cost
• Start simple, measure, then optimize

Test Your Knowledge

JSON Mode, schemas, retry: structured output under control?

1. JSON Mode vs Custom Structured Outputs: what is the difference?

Answer: JSON Mode guarantees syntactically valid JSON; Custom Structured Outputs additionally enforce your exact schema—fields, types, structure: the complete contract.

2. What does client.chat.parse() provide with Pydantic?

Answer: The full loop: the Pydantic schema declares the structure, the API constrains generation, and the response arrives already validated and typed as a Python object—zero manual parsing.

3. And on the TypeScript side?

Answer: The same comfort with Zod: the Zod schema defines and validates the structure—a single source of truth for generation and typing.

4. What to do when the output does not respect the schema?

Answer: Validate systematically, and plan a retry strategy: re-request with the error in context, or degrade gracefully—never blind trust in the output.

5. When is structured output indispensable?

Answer: As soon as a program consumes the response without human review: extraction, integrations, pipelines—the format ceases to be a courtesy, it becomes an interface.

Declared schema, constrained generation, output validation: the triad that makes AI pluggable into real software.