Ground Responses in Sources and Reduce Hallucinations

The hallucination problem in production

A hallucination is a factually incorrect response delivered with the confidence of a correct one. That confidence is exactly what makes it dangerous. A support chatbot that invents a procedure sends users down a dead end; a legal assistant that cites a non-existent statute exposes your client to risk; a medical tool that recommends the wrong dosage creates real harm. In all three cases, nothing in the response format signals the problem.

Grounding addresses this risk by constraining the model to base its answers exclusively on verified sources. Combined with the citation system covered in previous lessons, it creates a multi-layered safety net: the prompt defines expected behavior, citation parsing enables verification, and the pipeline decides how to handle responses that fail the check.

The system prompt: your first line of defense

A vague prompt produces vague grounding. You must state the rules like a specification: define the scope of sources, provide the exact phrase to use when information is missing, forbid filling gaps with general knowledge, require citations for every claim, and specify how to handle contradictions between sources.

system_prompt = """
You are a specialized documentation assistant.

ABSOLUTE RULES:
1. Answer ONLY using the sources provided via references.
2. If the information is not in the sources, respond:
   "This information is not available in the provided sources."
3. NEVER supplement with your general knowledge.
4. Cite every claim with the corresponding reference.
5. If two sources contradict each other, mention both versions.
"""

This prompt does not guarantee zero hallucinations, and you must be clear about this with your stakeholders: it significantly reduces their frequency. Mistral models, trained for grounding, follow these instructions with a high compliance rate. However, a high rate is not a 100% rate, which is why the following steps are necessary.

Verify grounding after generation

The validate_grounding function turns an intuition—“does this response look sourced?”—into a metric. It iterates through the response chunks, summing the text length on one side and counting citations and distinct sources on the other. It then derives a density expressed as citations per thousand characters. A 2,000-character response with only one reference immediately becomes suspicious, whereas a human reviewer might have missed it.

from mistralai.client.models import TextChunk, ReferenceChunk


def validate_grounding(response, min_citations=1):
    """Verifies that the response is grounded in sources."""
    content = response.choices[0].message.content

    text_length = 0
    citation_count = 0
    cited_refs = set()

    for chunk in content:
        if isinstance(chunk, TextChunk):
            text_length += len(chunk.text)
        elif isinstance(chunk, ReferenceChunk):
            citation_count += 1
            cited_refs.update(chunk.reference_ids)

    # Checks
    checks = {
        "has_citations": citation_count >= min_citations,
        "citation_count": citation_count,
        "unique_sources": len(cited_refs),
        "text_length": text_length,
        "citations_per_1000_chars": round(
            citation_count / max(text_length / 1000, 1), 1
        )
    }

    checks["is_grounded"] = checks["has_citations"]
    return checks


result = validate_grounding(final_response, min_citations=2)
if not result["is_grounded"]:
    print("ALERT: Response potentially ungrounded!")
    print(f"  Citations found: {result['citation_count']}")

Identify claims outside the scope

Counting citations does not tell you whether the text between them remains faithful to the sources. detect_ungrounded_claims tackles this blind spot with a simple lexical heuristic: it splits the response into sentences, compares words longer than five characters in each sentence against the concatenated snippets, and flags any sentence where less than 30% of the keywords appear in the sources. For a documentation assistant, the sentences that rise to the top of the suspect list are typically the “general knowledge” filler added by the model.

def detect_ungrounded_claims(response_text, source_snippets):
    """Detects sentences that do not match any snippet."""
    import re

    # Split response into sentences
    sentences = re.split(r'[.!?]+', response_text)
    sentences = [s.strip() for s in sentences if len(s.strip()) > 20]

    # Concatenate all snippets into a single text
    all_sources = " ".join(source_snippets).lower()

    suspicious = []
    for sentence in sentences:
        # Extract significant keywords (> 5 chars)
        keywords = [
            w.lower() for w in sentence.split()
            if len(w) > 5
        ]
        # Check coverage
        matches = sum(1 for kw in keywords if kw in all_sources)
        coverage = matches / max(len(keywords), 1)

        if coverage < 0.3:  # Less than 30% of keywords found
            suspicious.append({
                "sentence": sentence,
                "coverage": round(coverage, 2)
            })

    return suspicious

Assemble a trust pipeline

In production, these checks are not just analysis scripts: they become a gate the response must pass through before reaching the user. The GroundedRAG class implements this gate. It generates the cited response, validates it, and in strict mode, outright rejects responses with insufficient grounding, returning an explicit status rather than dubious text. The confidence score aggregates three signals—presence of citations, source diversity, and density—so your interface can display a reliability level or route low-confidence cases for human review.

class GroundedRAG:
    """RAG pipeline with grounding and validation."""

    def __init__(self, client, model="mistral-large-latest"):
        self.client = client
        self.model = model

    def query(self, question, references, strict=True):
        """Executes a RAG query with grounding validation."""

        # 1. Generate response with citations
        response = self._generate_with_citations(
            question, references
        )

        # 2. Validate grounding
        grounding = validate_grounding(response, min_citations=1)

        # 3. In strict mode, reject ungrounded responses
        if strict and not grounding["is_grounded"]:
            return {
                "status": "rejected",
                "reason": "Response insufficiently grounded in sources",
                "grounding_score": grounding
            }

        # 4. Return validated response
        return {
            "status": "ok",
            "response": response,
            "grounding_score": grounding,
            "confidence": self._compute_confidence(grounding)
        }

    def _compute_confidence(self, grounding):
        """Calculates a confidence score based on grounding."""
        score = 0.0
        if grounding["has_citations"]:
            score += 0.4
        if grounding["unique_sources"] >= 2:
            score += 0.3
        if grounding["citations_per_1000_chars"] >= 1.0:
            score += 0.3
        return round(score, 2)

Monitor grounding over time

A grounded pipeline on day one of production can degrade over time because the documentation base ages, because a prompt was tweaked, or because traffic shifts to poorly covered topics. The five metrics below calculate directly from the outputs of validate_grounding and deserve a permanent dashboard: it is their drift, more than their absolute value, that alerts you.

Key takeaways

• Grounding constrains the model to answer only from verified sources
• A strict system prompt is the first line of defense against hallucinations
• Post-generation validation programmatically verifies the grounding of every response
• Combine these strategies in a multi-level trust pipeline
• Continuously monitor grounding metrics to detect degradation