Extract Structured Metadata

Update November 2, 2026 — OCR. OCR 4.0 (mistral-ocr-4-0) was retired on September 30, 2026. Use OCR 4.1 (mistral-ocr-4-1), generally available since August 31 at the same price, or the alias mistral-ocr-latest, which has pointed to it since July 16. OCR 4.1 adds confidence scores at the block level. Addendum November 3, 2026: A local PDF is transmitted in base64 as {"type": "document_url", "document_url": "data:application/pdf;base64,…"}; the document_base64 type from older examples exists neither in the documentation nor in the mistralai SDK, and those examples would have failed. Sources: OCR Processor · Document QnA. Corrected November 3, 2026: The code previously read page.document_annotation page by page; in mistralai SDK 3.7.0, document annotation is a single field in the response (response.document_annotation), and image annotation is read from image.image_annotation.

Extract exactly what you need

Document annotation isn’t just about retrieving raw text. It produces structured, typed, and validated data ready for immediate use in your systems: a row inserted into a database, a field compared against a reference, or an amount verified before payment. The challenge shifts entirely to schema design, which is the focus of this lesson.

Structure instead of flattening

An extraction schema should reflect the document’s reality. An invoice comes from a legal entity with a company name, a SIRET number (French business ID), and an address. Instead of aligning eight fields like emetteur_rue, emetteur_ville, emetteur_siret, define an Adresse model, then an LegalEntity model that contains it. Reuse these building blocks for both the issuer and the recipient. The TypeDocument enumeration plays a similar role: it prevents the model from inventing a category and guarantees a value you can use in an if statement or for routing. Optional fields explicitly state what can be absent, while default values—such as default="France" for the country—spare the model from guessing the obvious.

from pydantic import BaseModel, Field
from typing import Optional
from enum import Enum

class DocumentType(str, Enum):
    INVOICE = "facture"
    CONTRACT = "contrat"
    REPORT = "rapport"
    FORM = "formulaire"
    OTHER = "autre"

class Address(BaseModel):
    street: Optional[str] = Field(None, description="Numéro et street")
    postal_code: Optional[str] = Field(None, description="Code postal")
    city: Optional[str] = Field(None, description="Ville")
    country: str = Field(default="France", description="Pays")

class LegalEntity(BaseModel):
    name: str = Field(..., description="Raison sociale")
    siret: Optional[str] = Field(None, description="Numéro SIRET")
    address: Optional[Address] = Field(None, description="Address postale")
    email: Optional[str] = Field(None, description="Address email")

class DocumentExtraction(BaseModel):
    document_type: DocumentType = Field(
        ..., description="Type du document"
    )
    date: Optional[str] = Field(
        None, description="Date principale (YYYY-MM-DD)"
    )
    emetteur: LegalEntity = Field(
        ..., description="Entité qui émet le document"
    )
    destinataire: Optional[LegalEntity] = Field(
        None, description="Entité destinataire"
    )
    resume: str = Field(
        ..., description="Résumé en 2-3 phrases du contenu"
    )
    langue: str = Field(
        ..., description="Langue du document (code ISO 639-1)"
    )

Repeating data

Many documents contain series: billing lines, clauses, or expense items. Model these as a list[...] of a dedicated sub-model. The model then scans the document table and produces as many LigneFacture objects as it finds rows, including the description, quantity, unit price, and total for each. This gives you enough data to recalculate the subtotal yourself and verify the document’s consistency before accepting it.

from pydantic import BaseModel, Field

class InvoiceLine(BaseModel):
    description: str = Field(..., description="Description du produit/service")
    quantity: float = Field(..., description="Quantité")
    unit_price: float = Field(..., description="Prix unitaire HT")
    amount: float = Field(..., description="Montant total HT de la ligne")

class DetailedInvoice(BaseModel):
    supplier: str = Field(..., description="Nom du supplier")
    number: str = Field(..., description="Numéro de facture")
    date: str = Field(..., description="Date (YYYY-MM-DD)")
    lines: list[InvoiceLine] = Field(
        ..., description="Lignes de la facture"
    )
    subtotal_excl_tax: float = Field(..., description="Sous-total HT")
    vat: float = Field(..., description="Montant TVA")
    total_incl_tax: float = Field(..., description="Total TTC")
    payment_terms: str = Field(
        ..., description="Conditions de paiement mentionnées"
    )

Checkboxes and forms

Scanned paper forms are where the model shines. A bool field paired with the question label is sufficient: the model recognizes whether a box is checked, extracts the corresponding question, and—crucially for medical or administrative use—infers that a field is missing rather than inventing a value, as the Mistral team demonstrated. The note “null if not provided” in the phone number description isn’t decorative; it explicitly authorizes an empty value.

from pydantic import BaseModel, Field

class FormResponse(BaseModel):
    question: str = Field(..., description="Intitulé de la question")
    answer: bool = Field(..., description="True si coché, False sinon")

class MedicalForm(BaseModel):
    patient_name: str = Field(..., description="Nom complet du patient")
    age: Optional[int] = Field(None, description="Âge du patient")
    sex: Optional[str] = Field(None, description="Sexe (M/F)")
    patient_id: Optional[str] = Field(None, description="ID patient")
    phone: Optional[str] = Field(
        None, description="Numéro de téléphone (null si non fourni)"
    )
    responses: list[FormResponse] = Field(
        ..., description="Réponses aux questions du formulaire"
    )

End-to-end implementation

Take the detailed invoice example and connect it to a base64-encoded document. The annotation prompt complements the schema by specifying the file type and emphasizing attention to billing lines; Pydantic validation on return converts the JSON string into an object you can traverse safely.

import os
import json
from mistralai.client import Mistral
from mistralai.extra import response_format_from_pydantic_model

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

ocr_response = client.ocr.process(
    model="mistral-ocr-latest",
    document={
        "type": "document_url",
        "document_url": f"data:application/pdf;base64,{document_base64}"
    },
    document_annotation_format=response_format_from_pydantic_model(
        DetailedInvoice
    ),
    document_annotation_prompt=(
        "Ce document est une invoice. Extrais toutes les lines "
        "de facturation avec leurs montants."
    )
)

# Extraction and validation
if ocr_response.document_annotation:
    invoice = DetailedInvoice.model_validate_json(
        ocr_response.document_annotation
    )

    print(f"Facture {invoice.numero} - {invoice.fournisseur}")
    print(f"Date : {invoice.date}")
    print(f"\nLignes :")
    for line in invoice.lines:
        print(f"  - {line.description}: {line.quantite} x "
              f"{line.prix_unitaire}€ = {line.montant}€")
    print(f"\nTotal TTC : {invoice.total_incl_tax}€")

Poor extraction results almost always stem from the schema, rarely from the model. Review your descriptions one by one: they must be precise and unambiguous, since the model relies on them alone to understand your expectations. Write them in the language of your documents—French works perfectly for French invoices—and reserve enums for fields with limited values and sub-models for structures you will reuse across schemas.

Key takeaways

• Pydantic schemas enable extraction of typed and validated data
• Use complex types (lists, enums, nested models) for precise extractions
• The model handles optional fields and checkboxes reliably
• Field descriptions guide extraction: be precise and explicit
• Combine document_annotation_prompt with the schema for maximum context