Introduction to RAG

What Is RAG?

RAG (Retrieval-Augmented Generation) is an architecture that combines an information retrieval system with a large language model (LLM). Instead of relying solely on the model’s internal knowledge, RAG first searches for relevant information in a database, then injects that information into the prompt to generate a response grounded in real data.

Why has RAG become essential? Because LLMs have three fundamental limitations, and they all show up at the same moment: when someone asks a question about your company. First, frozen knowledge: a model only knows what it learned during training, so it knows nothing about your pricing update last month. Next, hallucinations: when asked about an internal procedure it doesn’t know, the model produces a plausible but completely wrong answer with total confidence. Finally, private data is out of reach by design: the model has no access to your internal documents, and you don’t want them ending up in a training corpus anyway.

RAG solves all three problems by adding a retrieval step before generation. The model stops being a memory to query and becomes a writing engine applied to documents you control.

RAG Architecture

The RAG pipeline breaks down into two main phases: an indexing phase done upfront, followed by a cycle executed for every question. The four steps below form the backbone of the upcoming lessons.

Only the first step requires significant compute time; the next three run in under a second. This is what makes the architecture viable for a service queried continuously: you encode your documentation once, then each question costs only one embedding call and one generation call.

RAG vs Fine-Tuning

RAG and fine-tuning solve similar problems but in different ways. The table below, written here as a Python structure, pits the two approaches against each other across five criteria.

# RAG vs Fine-Tuning Comparison

comparison = {
    "RAG": {
        "update": "Instant (add/remove documents)",
        "cost": "Low (no retraining)",
        "traceability": "Excellent (you know where the info came from)",
        "hallucinations": "Reduced (answers grounded in sources)",
        "use_cases": "Documentation, FAQs, knowledge bases",
    },
    "Fine-tuning": {
        "update": "Expensive (retraining required)",
        "cost": "High (GPUs, time, annotated data)",
        "traceability": "Low (knowledge baked into the model)",
        "hallucinations": "Possible (model may extrapolate)",
        "use_cases": "Writing style, output format, specific tasks",
    },
}

The “update” line often decides it. Fixing an error in a procedure means, with RAG, replacing a file and re-encoding a few chunks—a matter of minutes. With fine-tuning, the same fix requires rebuilding a dataset and rerunning training, with no guarantee the old version was truly forgotten. In practice, RAG is usually the best choice for integrating evolving factual knowledge, while fine-tuning is better for changing the model’s behavior or style. The two aren’t mutually exclusive: a model fine-tuned to respond in your business’s expected format can still be fed by RAG.

The Tech Stack

Building a RAG pipeline with Mistral requires just three libraries, and installation takes one line.

# Essential dependencies
# pip install mistralai faiss-cpu numpy

from mistralai.client import Mistral  # LLM + Embeddings
import faiss                    # Vector store
import numpy as np              # Vector calculations

The mistralai SDK handles both model calls on its own: embeddings with mistral-embed and generation with mistral-large. faiss-cpu, Meta’s library for fast vector search, takes over once exhaustive comparison gets too slow—beyond a few tens of thousands of vectors. Finally, numpy handles the vector matrix operations that tie everything together. In the following lessons, you’ll build each step of this pipeline from scratch, from data preparation to response generation.

When to Use RAG?

RAG is especially well-suited when you have internal documents—technical documentation, procedures, contracts—that no public model has ever seen. It’s also the right choice when information changes frequently: prices, availability, regulations. A service that quotes outdated pricing costs more than one that doesn’t answer at all.

Three other situations point naturally to RAG. First, the need for traceability, when you must know where each piece of information came from—a common requirement in regulated sectors and, in any case, a condition for user trust. Second, the goal of reducing hallucinations, since a model with the right excerpt has no reason to invent. Finally, budget: when you lack the resources or annotated data for fine-tuning, RAG offers a practical path with just a handful of documents and a few hours of work.

Key Takeaways

• RAG combines information retrieval with LLM generation
• The architecture breaks down into: indexing, retrieval, augmentation, generation
• RAG is preferable to fine-tuning for evolving factual knowledge
• Mistral provides both essential building blocks: embeddings and LLM
• FAISS is the reference library for vector search