Skip to main content
EN · FR Start for free
Menu
Lesson 14 of 16 · Document AI: OCR and Document Intelligence

Multi-Document QnA

Lesson 14 of 16 Updated November 3, 2026 Developer
On this page
  1. When the Answer Spans Multiple Files
  2. Send Everything in a Single Call
  3. Separate Extraction from Querying
  4. Move to Document RAG
  5. An End-to-End Comparison Workflow
  6. Key Takeaways

Updated November 2, 2026 — OCR. OCR 4.0 (mistral-ocr-4-0) was retired on September 30, 2026. Use OCR 4.1 (mistral-ocr-4-1), which has been generally available since August 31 at the same price, or the alias mistral-ocr-latest, which has pointed to it since July 16. OCR 4.1 adds confidence scores at the block level. Addendum November 3, 2026: A local PDF is transmitted in base64 as {"type": "document_url", "document_url": "data:application/pdf;base64,…"}; the document_base64 type from older examples exists neither in the documentation nor in the mistralai SDK, and those examples would have failed. Sources: OCR Processor · Document QnA. Corrected November 3, 2026: The code previously read page.document_annotation page by page; in mistralai SDK 3.7.0, the document annotation is a single field in the response (response.document_annotation), and an image’s annotation is read from image.image_annotation.

When the Answer Spans Multiple Files

In many professional scenarios, the answer to a question isn’t found in a single document. You may need to compare contracts, cross-reference invoices with purchase orders, or analyze a series of reports. Three strategies exist, and the best approach is to choose based on volume rather than immediately jumping to the most sophisticated option.

Send Everything in a Single Call

The most direct method is to stack multiple document_url blocks in the same message. The model sees both contracts side by side and can genuinely compare them, something page-by-page processing cannot achieve. We switch to mistral-large-latest here because comparison involves reasoning, not just reading.

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": (
                        "Compare the payment terms "
                        "of these two contracts."
                    )
                },
                {
                    "type": "document_url",
                    "document_url": "https://example.com/contract-A.pdf"
                },
                {
                    "type": "document_url",
                    "document_url": "https://example.com/contract-B.pdf"
                }
            ]
        }
    ]
)

print(response.choices[0].message.content)

The limitation appears quickly: the sum of the documents must not exceed the model’s context window. Two ten-page brochures pass without difficulty; two hundred-page framework agreements do not.

Separate Extraction from Querying

For large volumes, the recommended strategy decouples these two steps. You pass each document through OCR, keep the resulting Markdown, and then manually construct the context sent to the LLM. The benefit is control: the truncation to 5,000 characters visible in the example, keeping each document name as a section header, and the system message that establishes the role of a legal assistant. You decide exactly what enters the context window instead of letting it happen automatically.

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

def extract_text(url: str) -> str:
    """Extracts text from a document via OCR."""
    response = client.ocr.process(
        model="mistral-ocr-latest",
        document={"type": "document_url", "document_url": url}
    )
    return "\n\n".join(p.markdown for p in response.pages)


# OCR extraction for each document
documents = {
    "Contract A": extract_text("https://example.com/contract-A.pdf"),
    "Contract B": extract_text("https://example.com/contract-B.pdf"),
    "Contract C": extract_text("https://example.com/contract-C.pdf"),
}

# Build the prompt with context
context = "\n\n---\n\n".join(
    f"## {name}\n\n{text[:5000]}"  # Limit if necessary
    for name, text in documents.items()
)

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": (
                "You are a legal assistant. Analyze the "
                "provided documents and answer questions."
            )
        },
        {
            "role": "user",
            "content": (
                f"Here are the documents:\n\n{context}\n\n"
                f"Question: Which contract offers the best "
                f"early termination conditions?"
            )
        }
    ]
)

Move to Document RAG

Beyond a few dozen documents, no context window will suffice. You must then send only the relevant passages, which requires splitting the texts, vectorizing them with mistral-embed, and searching by similarity at query time. Chunking with a 200-character overlap prevents a sentence cut in half from losing its meaning between chunks, and top_k determines how many passages will ultimately accompany the question.

import os
from mistralai.client import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

def create_embeddings(text: str) -> list[float]:
    """Creates embeddings for a text."""
    response = client.embeddings.create(
        model="mistral-embed",
        inputs=[text]
    )
    return response.data[0].embedding


def chunker(text: str, size: int = 1000, overlap: int = 200) -> list[str]:
    """Splits text into chunks with overlap."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + size
        chunk = text[start:end]
        chunks.append(chunk)
        start = end - overlap
    return chunks


# Simplified RAG pipeline
def index_document(name: str, text: str) -> list[dict]:
    """Indexes a document for search."""
    chunks = chunker(text)
    index = []

    for i, chunk in enumerate(chunks):
        embedding = create_embeddings(chunk)
        index.append({
            "document": name,
            "chunk_id": i,
            "text": chunk,
            "embedding": embedding
        })

    return index


def search(query: str, index: list[dict], top_k: int = 5) -> list[dict]:
    """Searches for the most relevant chunks."""
    q_embedding = create_embeddings(query)

    # Cosine similarity calculation
    import numpy as np
    scores = []
    for item in index:
        score = np.dot(q_embedding, item["embedding"])
        scores.append((score, item))

    scores.sort(key=lambda x: x[0], reverse=True)
    return [item for _, item in scores[:top_k]]

Let’s summarize the selection logic. Sending multiple documents in a single call remains unbeatable for simplicity and works well for two or three short documents. Separate OCR followed by QnA provides flexibility and gives you control over what is sent to the LLM, at the cost of some extra code. Document RAG is heavier to set up but becomes essential as the corpus grows, because it is the only approach that scales.

An End-to-End Comparison Workflow

The two techniques can be combined: nothing prevents you from first extracting structured data via annotation, then handing that data to the LLM for analysis. On a batch of invoices, each document is reduced to three fields—vendor, amount, date—before a single request asks the model to identify trends and anomalies. The token cost becomes negligible, and the analysis focuses on cleanly extracted figures rather than text interpreted on the fly.

import os
from pydantic import BaseModel, Field
from mistralai.client import Mistral
from mistralai.extra import response_format_from_pydantic_model

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

class InvoiceSummary(BaseModel):
    vendor: str = Field(..., description="Vendor name")
    amount_inc_tax: float = Field(..., description="Amount including tax")
    date: str = Field(..., description="Date (YYYY-MM-DD)")

def compare_invoices(paths: list[str]) -> str:
    """Compares multiple invoices and returns an analysis."""
    import base64

    invoices = []
    for path in paths:
        with open(path, "rb") as f:
            doc_b64 = base64.b64encode(f.read()).decode("utf-8")

        response = client.ocr.process(
            model="mistral-ocr-latest",
            document={
                "type": "document_url",
                "document_url": f"data:application/pdf;base64,{doc_b64}"
            },
            document_annotation_format=response_format_from_pydantic_model(
                InvoiceSummary
            )
        )

        if response.document_annotation:
            invoice = InvoiceSummary.model_validate_json(
                response.document_annotation
            )
            invoices.append(invoice)

    # Comparative analysis via LLM
    summary = "\n".join(
        f"- {f.vendor}: €{f.amount_inc_tax} ({f.date})"
        for f in invoices
    )

    response = client.chat.complete(
        model="mistral-small-latest",
        messages=[
            {
                "role": "user",
                "content": (
                    f"Here are {len(invoices)} invoices:\n{summary}\n\n"
                    f"Analyze trends and anomalies."
                )
            }
        ]
    )

    return response.choices[0].message.content

Key Takeaways

  • Three approaches for multi-document QnA: inline, separate OCR, RAG
  • The inline approach works for 2–3 short documents
  • Separate OCR + QnA offers more control over consumed tokens
  • RAG is essential for large corpora (dozens of documents)
  • Combine structured annotations with QnA for precise analyses
← Previous lessonThe Document QnA APINext lesson →OCR + RAG Integration