Introduction to Document AI

Updated November 2, 2026 — OCR. OCR 4.0 (mistral-ocr-4-0) was retired on September 30, 2026. Use OCR 4.1 (mistral-ocr-4-1), which has been generally available since August 31 at the same price, or the alias mistral-ocr-latest, which has pointed to it since July 16. OCR 4.1 adds confidence scores at the block level.

Why Document AI Changes the Game

Every morning, an insurance company receives hundreds of claim forms: scanned documents tilted at odd angles, handwritten reports filled out with a pen, and photos of invoices taken on a phone. Banks face the same issue with statements, hospitals with patient records, and government agencies with forms. For decades, traditional optical character recognition (OCR) has tried to solve this problem, often with disappointing results as soon as layouts get complex, handwriting is involved, or documents mix multiple languages.

Mistral AI built Document AI to handle exactly this kind of workflow. It isn’t just another OCR engine: the solution combines a specialized vision language model (VLM) with structured extraction and document intelligence capabilities. In other words, it doesn’t just read characters; it reconstructs the document as it’s organized and lets you query it afterward.

The Three Document AI Services

Document AI relies on three complementary services, all accessible via the Mistral API. It helps to understand how they chain together right from the start.

The first, the OCR Processor, is the core of the system. The mistral-ocr-latest model (OCR 4) extracts text from scanned documents, PDFs, images, and presentations. Where a classic OCR like Tesseract gives you a jumbled mess of lines, this model preserves structure: headings, paragraphs, tables, lists, and hyperlinks. Take that insurance claim form again: you get the form back with its sections identified, not a block of text you have to untangle.

The second service, Document Annotations, takes over to extract actionable data. You define a Pydantic schema—patient name, invoice amount, contract clauses—and the model fills these fields automatically. You move from readable text to typed values that an information system can ingest.

The third, Document QnA, combines OCR with a Mistral LLM to let you ask questions in natural language about the content. This is what lets a lawyer ask, “What is the notice period for termination?” instead of rereading forty pages. It’s also the natural way to access large archives or compare reports.

Traditional OCR vs. Document AI

Traditional OCR, whether Tesseract or Adobe Acrobat, works by recognizing patterns pixel by pixel. That principle works fine on a crisp printed page, but it falls apart as soon as you step outside that ideal case. A newspaper with multiple columns, nested headers, and illustrated sidebars produces text where the reading order no longer makes sense. Handwritten doctor’s notes or manually filled forms remain illegible to it. Tables with merged cells, missing borders, or irregular alignments lose all relationship between columns and rows. Add a document mixing Latin, Arabic, and Asian scripts, or a noisy, tilted, blurry scan, and the error rate becomes prohibitive.

Document AI builds on a fine-tuned vision model (VLM) that understands document semantics. It doesn’t just recognize characters: it grasps structure, context, and relationships between elements. This fundamental difference in nature, not just a simple gain in accuracy, explains its results on difficult cases.

Structured Markdown Output

Another key difference: Document AI produces structured Markdown, not plain text. Each page is returned with its textual content tagged—headings, lists, paragraphs—its tables formatted in Markdown or HTML, extracted images accompanied by their position, preserved hyperlinks, and, if you wish, headers and footers isolated from the body. This output is ready to use: you can index it in a vector database for RAG, archive it, or display it as is, without an intermediate cleaning step.

On the input side, the service covers PDFs (scanned and native), PNG, JPEG, and AVIF images, PPTX presentations, and DOCX documents. The maximum file size is 50 MB, with a cap of 1,000 pages—two limits to keep in mind when splitting your batches.

What You Will Learn

The course follows the order of the three services. Lessons 1 to 3 cover the overview, the OCR 4 model, and use cases. Lessons 4 to 8 dive deeper into OCR: the API, Python code, tables, headers and footers, and accuracy optimization. Lessons 9 to 12 handle annotations, from metadata extraction to batch processing. Lessons 13 to 16 tackle document QnA, from single documents to multi-document setups, through to RAG integration and production pipelines.

Key Takeaways

• Document AI combines OCR, structured annotations, and QnA in a single solution
• The OCR 4 model far surpasses traditional OCR solutions in accuracy and flexibility
• Structured Markdown output makes integration into your existing pipelines easy
• Three complementary services cover the entire document lifecycle