Grounding: Anchor Responses in Facts

Lesson updated November 3, 2026 based on OpenAI documentation: temperature and top_p are only accepted with a reasoning effort of none, whereas GPT-5.6 models reason by default (medium). Calls that set these parameters must therefore specify reasoning={"effort": "none"} (Responses API) or reasoning_effort="none" (Chat Completions). Sources: Using GPT-6 · GPT-5.6 Terra.

Grounding: Anchor Responses in Facts

Grounding constrains the model to base its answers solely on data you provide, rather than on its internal knowledge. It is your first line of defense against hallucinations and a non-negotiable requirement once an application leaves the prototype stage. A support assistant that invents a contract clause isn’t making a cute mistake: it legally binds your company to the client who read it.

Why Grounding Is Essential

LLMs generate statistically probable text, which does not necessarily align with factually correct text. Without grounding, a model can fabricate citations, statistics, or technical references that sound perfectly plausible: a version number that never existed, a credible but false cooling-off period, or a parameter name invented out of thin air. Nothing in the phrasing gives away the fabrication, which is precisely what makes the phenomenon dangerous in production. The better written the response, the less likely the user is to verify it, and the higher the legal and reputational risk becomes.

Technique 1: Explicit Context

The simplest method is also the most robust: inject the source text into the prompt and forbid the model from going beyond it. Your instructions must include three elements: a restriction to the provided document, the exact fallback phrase to use when information is missing, and a requirement to cite the passage used. Test the result on an edge case: a refund request at 45 days must trigger the 50% partial refund rule, not the accommodating answer the customer hopes to hear.

from openai import OpenAI

client = OpenAI()

document = """
Acme Corp Refund Policy (v3.2, January 2026):
- Full refund within 30 days of purchase
- Partial refund (50%) between 30 and 60 days
- No refund after 60 days
- Customized products are non-refundable
- Refunds are processed within 5 business days
"""

system = """You are an Acme Corp customer support assistant.

STRICT RULES:
- Answer ONLY based on the document provided below
- If the answer is not in the document, say "I do not have
  this information in our current policy"
- NEVER make assumptions beyond the document
- Quote the relevant passage in quotation marks

REFERENCE DOCUMENT:
""" + document

response = client.responses.create(
    model="gpt-5.6-terra",
    reasoning={"effort": "none"},  # temperature requires effort none
    instructions=system,
    input="I would like a refund for an item purchased 45 days ago.",
    temperature=0.1
)

Technique 2: Mandatory Citations

The next level up requires a reference for every assertion made. This has a dual effect. The response becomes verifiable by the user, who can trace it back to the original passage instead of taking your word for it. And the model self-censors on claims it cannot tie to any source, since the instruction explicitly tells it to omit them rather than deliver them without reference. Numbering the documents in the context, as the function below does, also makes the citations usable in your interface: “[Document 2]” can become a clickable link to the correct file.

system_with_citations = """You are a research assistant.

For every assertion in your response, add a reference
to the source passage in brackets, for example: [Source: paragraph 3].

If you cannot cite a source passage, do not include the assertion.

Response format:
1. Response with inline citations
2. "Sources Used" section with exact excerpts"""

def reponse_grounded(question: str, documents: list[str]) -> str:
    """Generate a grounded response with citations."""
    context = "\n\n---\n\n".join(
        f"[Document {i+1}]\n{doc}"
        for i, doc in enumerate(documents)
    )

    response = client.responses.create(
        model="gpt-5.6-terra",
        reasoning={"effort": "none"},  # temperature requires effort none
        instructions=system_with_citations + f"\n\nDOCUMENTS:\n{context}",
        input=question,
        temperature=0.1
    )
    return response.output_text

Technique 3: Post-Generation Verification

The first two techniques reduce risk without ever eliminating it. For critical cases, add a second pass: a dedicated call that breaks the response down into elementary assertions and checks each one against the sources. The principle is the same as code review—the author reads their own work poorly, while a fresh pair of eyes reads it much better—and it works here for the same reason, since the verifier did not produce the text it is examining. Setting temperature to 0.0 is not a detail: a verifier must be as reproducible as possible, otherwise you won’t know whether a changing verdict comes from the response or the model’s mood.

def verifier_grounding(reponse: str, sources: str) -> dict:
    """Verify whether a response is properly grounded in the sources."""
    verification_prompt = f"""Analyze whether this response is faithful to the sources.

RESPONSE TO VERIFY:
{reponse}

SOURCES:
{sources}

For each assertion in the response, indicate:
- The assertion
- Whether it is supported by the sources (YES/NO)
- The corresponding source passage (or "None")

End with a grounding score: X supported assertions / Y total."""

    response = client.responses.create(
        model="gpt-5.6-terra",
        reasoning={"effort": "none"},  # temperature requires effort none
        input=verification_prompt,
        temperature=0.0
    )
    return {"verification": response.output_text}

Technique 4: Defensive Anti-Hallucination Prompting

Certain phrasings significantly reduce hallucinations, and they are not interchangeable: each addresses a distinct symptom. The explicit formulation simply allows the model to admit ignorance, which is often enough. The penalizing approach ranks errors against each other, useful when facing a model that spontaneously prefers to say anything rather than nothing. The structural approach surfaces uncertainty in the output itself, valuable when your interface needs to visually distinguish what is certain from what is inferred. Finally, the constraint approach delimits the source with tags, which works particularly well when the injected context is voluminous and the boundary between document and instruction risks blurring.

defensive_prompts = {
    "explicit": "If you do not know the answer, say 'I don't know'.",

    "penalizing": "It is FORBIDDEN to invent information. "
                  "An 'I don't know' response is always preferable "
                  "to incorrect information.",

    "structural": "For each point in your response, indicate your confidence level: "
                  "[CERTAIN] based on the document, "
                  "[PROBABLE] inferred from context, [UNCERTAIN] unverified.",

    "constrained": "Answer using EXCLUSIVELY the information "
                   "between the <source> and </source> tags. "
                   "Any external information is forbidden."
}

When used correctly, these phrasings reduce hallucinations by 60 to 80%. However, you need to observe this in your own environment rather than relying on an average: the grounding rate is calculated by replaying verification on a batch of responses and aggregating the scores obtained.

def taux_grounding(reponses: list[dict]) -> float:
    """Calculate the grounding rate across a set of responses.

    Each response is a dict with 'reponse' and 'sources'.
    """
    scores = []
    for item in reponses:
        verif = verifier_grounding(item["reponse"], item["sources"])
        # Extract the score (adapt to output format)
        scores.append(verif)
    return scores

Combining with API Tools

The OpenAI API offers two native tools that spare you from building your own retrieval layer. File Search looks through documents you have uploaded and automatically injects relevant excerpts into the context. Web Search anchors responses in recent web results, which helps when the subject evolves faster than your document base. In both cases, restriction instructions remain essential: the tool handles bringing in the material, but your prompt—and your prompt alone—forbids the model from straying from it.

response = client.responses.create(
    model="gpt-5.6-terra",
    instructions="Answer based only on the provided documents.",
    input="What are the warranty conditions?",
    tools=[{"type": "file_search"}]
)

Now put the full chain to the test on an internal company document—an FAQ, terms of service, technical documentation, it doesn’t matter as long as it’s real. Build the grounded assistant using explicit context, query it with ten questions where three deliberately cover points absent from the document, and verify that it refuses those three instead of embellishing. Then add post-generation verification and measure your actual grounding rate: it is this figure, not the confidence you place in your prompt, that decides whether the assistant is ready for production.

Key Takeaways

• Grounding is essential for any professional application
• Inject source context directly into the prompt and enforce citations
• Verify grounding after generation for critical cases
• Defensive phrasings reduce hallucinations by 60 to 80%
• Use temperature=0.0-0.1 to maximize fidelity to sources