Multi-turn conversations: keeping reasoning in the history

Lesson rewritten on November 2, 2026. It previously covered Magistral’s prompt_mode parameter, which was removed with that model. It now addresses a rule highlighted in the current documentation: in multi-turn conversations, reasoning must remain in the history. Source: Reasoning — Multi-turn conversations.

The rule: replay the full message

With the Chat Completions API, you are responsible for sending the conversation history at every turn. The temptation is strong to include only the response text, which is shorter and more readable. Mistral’s documentation is explicit: always replay the assistant’s full message, including reasoning fragments. Stripping the reasoning trace from one turn to the next degrades model performance, as the model relies on it to maintain coherence. Saving tokens comes at the cost of quality.

Reference code

The simplest approach is to add the message object returned by the SDK to the history exactly as is:

import os
from mistralai.client import Mistral
from mistralai.client.models import TextChunk, UserMessage

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"], timeout_ms=300_000)
messages = []

for question in ["What is 17 × 23?", "Now multiply that result by 3."]:
    messages.append(UserMessage(content=question))

    response = client.chat.complete(
        model="mistral-large-4-0",
        messages=messages,
        reasoning_effort="high",
    )
    assistant_message = response.choices[0].message

    # Display: response text only
    content = assistant_message.content
    if isinstance(content, str):
        print(content)
    else:
        print("".join(c.text for c in content or [] if isinstance(c, TextChunk)))

    # History: full message, including reasoning
    messages.append(assistant_message)

The timeout_ms parameter increases the client’s wait time, which is useful when long reasoning extends the response time. The critical point is the last line: you do not add a message reconstructed from the text alone, but assistant_message itself.

The cost

The official example illustrates the impact on tokens. On the first turn, the request contains 25 input tokens: just the question. The model responds with 187 tokens, including reasoning. On the second turn, the input rises to 221 tokens: the first question (25), the full first-turn response with its reasoning (187), and the new question (9). Output tokens from one turn become input tokens on the next, and this is precisely what allows the model to know unambiguously which “result” the user is referring to.

Over a long conversation, this mechanism causes the history to grow with each turn. Two practical consequences: monitor usage.prompt_tokens, which grows with the conversation, and keep the model’s context window in mind (256,000 tokens for Small 4, one million for Large 4). If the history becomes too heavy, start a new conversation with a summary of the previous one rather than stripping out reasoning traces.

Streaming and the Conversations API

In streaming mode, the response arrives in chunks (Lesson 3): for the history, you must therefore reassemble a complete message yourself, including reasoning, which complicates the code. If your application is conversational, the Conversations API offers an alternative: the history is maintained server-side, and you continue the exchange with conversations.append without resending previous turns (Lesson 4).

Histories generated by Magistral

If you are preserving conversations generated by Magistral, note that their format has varied. Versions 25.07 and 25.09 already separated reasoning into thinking fragments, as they are today. Version 25.06, the first release, placed the reasoning directly in the response text, between <think> and “ tags. Before replaying such histories on Small 4 or Large 4, test resumption on a few conversations: there is no guarantee that a current model will leverage reasoning written in a different format by another model.

Key takeaways

• In Chat Completions, replay the assistant’s full message, including reasoning
• Stripping reasoning fragments saves tokens but significantly degrades quality
• Output tokens from one turn become input tokens on the next: monitor prompt_tokens
• In streaming, the Conversations API avoids the need to reassemble history manually
• Magistral 25.06 histories mixed reasoning and response in the text, using <think> tags