Stop sequences

Stopping generation at the right moment

By default, the model stops for only two reasons: it decides it has finished its response and emits its end token, or it hits the max_tokens ceiling. These two criteria don’t cover every use case. Sometimes you want generation to stop at a point only you know—just before the next section of a document, right after a JSON block closes, or as soon as the model starts adding commentary you don’t need.

Stop sequences add this third criterion: as soon as the model produces a text sequence you’ve designated, generation halts immediately. Think of it as a precise cutting tool, the natural complement to the formatting instructions in your system message, which the model follows well but not always perfectly.

How the stop parameter works

The parameter accepts a list of strings. The first one that appears in the output ends generation:

from mistralai.client import Mistral
import os

client = Mistral(api_key=os.getenv("MISTRAL_API_KEY"))

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {"role": "user", "content": "List the 10 largest cities in France."}
    ],
    stop=["6."]  # Stop after the first 5
)

print(response.choices[0].message.content)
# 1. Paris
# 2. Marseille
# 3. Lyon
# 4. Toulouse
# 5. Nice

Two behaviors stand out in this example. The stop sequence is not included in the response: you get the text that precedes it, with no cleanup needed on your end. And the finish_reason is "stop", exactly as with a natural stop—the API doesn’t distinguish between the two, so it’s up to you to remember what you requested.

Three typical use cases

The first case handles the chatty model. You ask for a capital city, and it gives you the capital plus three sentences of explanation your interface wasn’t expecting. The system prompt sets the intent; stop sequences enforce it:

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": "Answer with only the city name, nothing else."
        },
        {
            "role": "user",
            "content": "What is the capital of Italy?"
        }
    ],
    stop=["\n", ".", "Explanation"]  # Stop at the first newline or period
)

print(response.choices[0].message.content.strip())
# Rome

The second case covers section-by-section document generation, when you want to keep control between parts rather than letting the model produce an entire report in one go. You stop on the delimiters that signal what comes next:

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": "Write the Introduction section of a report."
        },
        {
            "role": "user",
            "content": "Topic: the impact of AI on employment in Europe."
        }
    ],
    stop=["## ", "---", "Conclusion"]  # Stop before the next section
)

The third case is the most cost-effective in practice: extracting usable JSON without a cleanup step. Models have a tendency to wrap their output in explanatory text, which breaks json.loads(); cutting at a double newline or triple backticks eliminates the problem at the source.

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {
            "role": "system",
            "content": "Respond only with valid JSON."
        },
        {
            "role": "user",
            "content": "Give me info for Paris: population, country, time zone."
        }
    ],
    stop=["\n\n", "```"]  # Stop after the JSON block
)

Combining stop sequences and streaming

Nothing prevents you from combining both mechanisms: the stream stops as soon as one of the stop sequences is detected in the generated text, and the chunks you’ve already received remain valid.

stream = client.chat.stream(
    model="mistral-large-latest",
    messages=[
        {"role": "user", "content": "Tell a short story."}
    ],
    stop=["Fin.", "THE END", "---"]
)

full_text = ""
for chunk in stream:
    content = chunk.data.choices[0].delta.content
    if content:
        full_text += content
        print(content, end="", flush=True)

print(f"\n\nText: {len(full_text)} characters")

Mechanism limitations

Four constraints govern the use of the stop parameter. You can provide at most four sequences per request, which forces you to choose the most discriminative ones rather than piling them on. They are case-sensitive: "end" will never catch "End", so you need to account for the variants that matter. The sequence is never included in the response, which matters if your expected format ends with that delimiter. Finally, detection happens token by token: if your sequence falls in the middle of a token, it goes unnoticed—another reason to choose multi-character delimiters rather than a single symbol.

Costly mistakes

The most common error is a sequence that’s too short, triggering premature stops. A lone period stops generation at the first sentence it sees, even in the middle of an abbreviation; the symmetric mistake is choosing a string the model will never produce, in which case the parameter does nothing and you mistakenly believe you’ve set a guardrail.

# BAD: sequence too short, premature stops
stop=["."]  # Stops at every sentence!

# GOOD: context-specific sequence
stop=[".\n\n"]  # Stops after a period followed by a double newline

# BAD: sequence the model will never produce
stop=["XYZZY"]  # Has no effect

# GOOD: realistic sequence based on expected format
stop=["Next question:", "---"]

The rule that follows from these four lines is simple: a good stop sequence derives from the format you asked the model to use, never from your imagination.

Complete example: controlled Q&A

Let’s assemble these elements into a short question-and-answer function, where low temperature ensures factual stability, max_tokens sets a safety ceiling, and stop sequences cut the commentary the model might be tempted to add after its answer:

from mistralai.client import Mistral
import os

client = Mistral(api_key=os.getenv("MISTRAL_API_KEY"))

def ask_single_question(question: str) -> str:
    """Ask a question and get a concise answer without chatter."""
    response = client.chat.complete(
        model="mistral-large-latest",
        messages=[
            {
                "role": "system",
                "content": "Answer in a single sentence, directly and factually."
            },
            {"role": "user", "content": question}
        ],
        max_tokens=100,
        stop=["\n\n", "Remark:", "Note:"],
        temperature=0.1
    )

    return response.choices[0].message.content.strip()

# Test
questions = [
    "What is the speed of light?",
    "Who invented the telephone?",
    "How many planets are in the solar system?"
]

for q in questions:
    print(f"Q: {q}")
    print(f"A: {ask_single_question(q)}\n")

Run this script once as is, then a second time after removing the stop parameter: comparing the two outputs shows you exactly what those three sequences filter out.

Key takeaways

• Stop sequences halt generation when the model produces specific text
• Maximum of 4 sequences per request; case-sensitive
• The stop sequence is never included in the response
• Use sequences specific enough to avoid premature stops
• Compatible with streaming and all other parameters