Few-shot via the API

Teach by example

Few-shot prompting means giving the model a few input–output examples before you send your actual request. With the Chat Completions API, these examples don’t get a special slot: you inject them into the conversation history as fake user/assistant message pairs, as if the exchange really happened. The model can’t tell the difference between a genuine turn and one you fabricated, and that’s exactly what makes the technique so effective. Without fine-tuning, this is the most reliable way to get a precise output format.

The trick is an inversion: instead of describing the expected format at length in the system prompt, you show it. For a sentiment classification task, three examples are enough to make the model understand it must reply with an uppercase label and nothing else.

from mistralai.client import Mistral
import os

client = Mistral(api_key=os.getenv("MISTRAL_API_KEY"))

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": "You classify the sentiment of customer reviews."
        },
        # Example 1
        {"role": "user", "content": "Review: Fast delivery, product as described. Very satisfied!"},
        {"role": "assistant", "content": "POSITIVE"},
        # Example 2
        {"role": "user", "content": "Review: Package arrived damaged, customer service unreachable."},
        {"role": "assistant", "content": "NEGATIVE"},
        # Example 3
        {"role": "user", "content": "Review: The product is fine but a bit pricey for what it is."},
        {"role": "assistant", "content": "MIXED"},
        # Actual request
        {"role": "user", "content": "Review: Excellent quality, I highly recommend this store."}
    ],
    temperature=0.0
)

print(response.choices[0].message.content)
# POSITIVE

The model extracted the pattern—review as input, uppercase label as output—and applied it to the new input. No formatting instruction was written out explicitly: the three pairs were enough.

How many examples to provide

The number of examples depends on the task difficulty, and you hit sharp diminishing returns.

Beyond ten examples, you burn a lot of input tokens without proportional gain: every request drags the same block of examples, billed on every call. At that point, fine-tuning becomes the economically sensible option.

Two uses that change the output

Named entity extraction shows what examples add beyond a schema described in full. They lock in not only the JSON keys, but also how to handle empty categories—the second example explicitly shows empty lists instead of a missing field, which keeps your parser from hitting an unexpected structure.

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": "Extract named entities in JSON format."
        },
        # Example 1
        {
            "role": "user",
            "content": "Pierre Martin joined Google France in Paris in January 2025."
        },
        {
            "role": "assistant",
            "content": '{"personnes": ["Pierre Martin"], "organisations": ["Google France"], "lieux": ["Paris"], "dates": ["janvier 2025"]}'
        },
        # Example 2
        {
            "role": "user",
            "content": "The VivaTech conference organized by Publicis will take place in June."
        },
        {
            "role": "assistant",
            "content": '{"personnes": [], "organisations": ["Publicis"], "lieux": [], "dates": ["juin"]}'
        },
        # Actual request
        {
            "role": "user",
            "content": "Sophie Leclerc, director at Mistral AI, announced a partnership with Microsoft at the AI Paris show on March 15, 2026."
        }
    ],
    temperature=0.0
)

import json
result = json.loads(response.choices[0].message.content)
print(json.dumps(result, indent=2, ensure_ascii=False))

Text transformation follows a different logic: you’re not transmitting a schema, but a register of language. A single example of simplification—translating transformer architecture into an accessible sentence—teaches the model the desired level of abstraction, something no instruction like “be pedagogical” would achieve with this precision.

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": "Transform technical text into a layperson summary."
        },
        # Example
        {
            "role": "user",
            "content": "The model uses a transformer encoder-decoder architecture with multi-head attention and pre-layer RMS normalization."
        },
        {
            "role": "assistant",
            "content": "This AI system uses advanced technology that lets it understand and generate text by analyzing relationships between words in parallel and efficiently."
        },
        # Actual request
        {
            "role": "user",
            "content": "The API uses nucleus sampling with top_p to truncate the probability distribution of candidate tokens during autoregressive generation."
        }
    ]
)

print(response.choices[0].message.content)

Dynamic few-shot

In production, a fixed set of examples ends up hurting half your requests: technical examples unnecessarily bloat a billing question. The solution is to build an example base by category, then inject only the ones that match the context detected upstream. The function below assembles the system prompt, the examples for the category, and the actual question in the right order.

# Example base
EXAMPLES = {
    "technical": [
        {"user": "How do I configure a VPN?", "assistant": "Here are the steps..."},
        {"user": "My server is returning a 502 error.", "assistant": "Error 502..."},
    ],
    "sales": [
        {"user": "What are your rates?", "assistant": "Our plans start at..."},
        {"user": "Is there a trial period?", "assistant": "Yes, 14 days..."},
    ],
    "billing": [
        {"user": "I haven't received my invoice.", "assistant": "Check your..."},
        {"user": "How do I change plans?", "assistant": "Go to..."},
    ]
}

def build_messages(category: str, user_question: str) -> list:
    """Build messages with dynamic few-shot."""
    messages = [
        {"role": "system", "content": "Technical support assistant."}
    ]

    # Add examples for the category
    for ex in EXAMPLES.get(category, []):
        messages.append({"role": "user", "content": ex["user"]})
        messages.append({"role": "assistant", "content": ex["assistant"]})

    # Add the actual question
    messages.append({"role": "user", "content": user_question})
    return messages

# Usage
messages = build_messages("technical", "My Docker deployment is failing.")
response = client.chat.complete(
    model="mistral-large-latest",
    messages=messages
)

What makes a good example set

Few-shot fails most often due to lack of diversity: three outright positive reviews and the model will classify everything as POSITIVE. Cover normal cases and edge cases, including an example of what the model must NOT do—showing a refusal is often more effective than forbidding it with a rule. Consistency in form matters just as much: if one example returns POSITIVE and another Positive, you’ve just taught the model that casing is free.

Two habits complete the exercise. Count tokens, since every example is billed as input on every call—an eight-pair set quickly becomes the main cost driver for a high-volume classification API. And test without examples first: recent models handle in zero-shot tasks that required five examples two years ago, and you’ll save what you didn’t need to write.

Key takeaways

• Few-shot injects fake user/assistant pairs before the actual request
• 3–5 examples are enough for most classification and extraction tasks
• Use temperature=0 with few-shot to maximize consistency
• Dynamic few-shot selects relevant examples based on context
• Beyond 10 examples, consider fine-tuning instead of few-shot