Latency Optimization

Course updated November 3, 2026 for Voxtral TTS 3, based on the Migrate from Voxtral 2 guide and Mistral’s Text to Speech pages: voxtral-mini-tts-2603, deprecated October 31, 2026 and retired April 30, 2027, is replaced in examples by voxtral-mini-tts-3-0. Voxtral 2 predefined voices (gb_jane_neutral…) remain supported; Voxtral 3 adds [emotion=…] and [pace=…] tags in text, the expressiveness setting, custom pronunciation (phoneme_hints), and 35 languages. Mistral recommends re-measuring latency after migration. Sources: Migrate from Voxtral 2 · Speech generation.

Latency: The Enemy of Natural Conversation

In human conversation, the average response time hovers around 200 milliseconds. Beyond 500 ms, the interlocutor perceives a delay; beyond one second, they assume you didn’t hear them and repeat their question. For a voice assistant to feel natural, the time between the end of the question and the start of the response must stay under one second.

Voxtral advertises a model latency of approximately 90 ms, and this is where many teams go wrong: that figure describes only one link in a much longer chain. What the user experiences is the sum of all links. Optimization therefore requires knowing the cost of each component.

Anatomy of Total Latency

The journey begins with the outbound network: the time it takes to route your request to Mistral’s servers. This depends on your location and connection quality; from Europe, expect 20 to 80 ms. Next comes model latency, those famous 90 ms of processing before the first audio fragment—the only part over which you have no control, since it belongs to Mistral’s infrastructure.

The next step is encoding, and this is where the nastiest surprises lurk. PCM costs nothing, or next to nothing, because there is nothing to encode. Opus adds 5 to 10 ms, which remains negligible. MP3, however, demands 200 to 500 ms: its codec must accumulate enough data before producing the first frame. In other words, a developer who leaves response_format="mp3" out of habit adds more delay on their own than the entire model does.

Finally, the return network weighs little in streaming since fragments are small. What remains is the client-side buffer: your application generally accumulates two or three fragments before starting playback to avoid micro-dropouts. Add 50 to 150 ms—a deliberate cost that buys smoothness.

Streaming vs. Batch: The Numbers

These orders of magnitude were recorded with Voxtral 2. Mistral announces approximately 90 ms before the first sound for Voxtral TTS 3 on the model side, and recommends re-measuring latency from your application after migration: redo these measurements with your network and your format.

Switching from batch to PCM streaming cuts perceived latency by a factor of three to four. No other optimization in this lesson will have a comparable effect: this is the first thing to change, even before measuring.

Three Server-Side Levers

The first lever is format. For any conversational application, streaming PCM is the reference setting, and the switch happens without rewriting your logic.

with client.audio.speech.complete(
    model="voxtral-mini-tts-3-0",
    input=text,
    voice_id=voice_id,
    response_format="pcm",  # Minimum latency
    stream=True,
) as stream:
    for event in stream:
        if event.event == "speech.audio.delta":
            play_audio(base64.b64decode(event.data.audio_data))

The second lever lies in the choice between a saved voice and on-the-fly cloning. A voice_id designates a voice already processed server-side, whereas ref_audio forces reprocessing the sample on every request. For an agent responding a hundred times an hour with the same voice, this detail saves 50 to 100 ms per turn of speech.

The third lever is to shorten what you request. The shorter the text, the faster the first fragment arrives: generating a five-sentence response in one block means making the user wait for content they won’t hear for another fifteen seconds. The following generator splits the response by sentence and produces audio for the first one while the others wait.

def generate_by_sentence(full_text, voice_id):
    """Generate audio sentence by sentence to minimize latency."""
    sentences = full_text.split(". ")

    for sentence in sentences:
        sentence = sentence.strip()
        if not sentence:
            continue
        if not sentence.endswith("."):
            sentence += "."

        with client.audio.speech.complete(
            model="voxtral-mini-tts-3-0",
            input=sentence,
            voice_id=voice_id,
            response_format="pcm",
            stream=True,
        ) as stream:
            for event in stream:
                if event.event == "speech.audio.delta":
                    yield base64.b64decode(event.data.audio_data)

Two Client-Side Levers

A fixed buffer is always a poorly placed compromise: too small, and it lets micro-dropouts slip through on a loaded network; too large, and it adds unnecessary delay on a good connection. The class below starts playback as soon as two fragments are available, then lets subsequent ones pass through as they arrive, never accumulating beyond five.

class AdaptiveBuffer:
    def __init__(self):
        self.buffer = []
        self.min_threshold = 2   # Minimum chunks before playback
        self.max_threshold = 5   # Do not accumulate more
        self.playback_started = False

    def add_chunk(self, chunk):
        self.buffer.append(chunk)

        if not self.playback_started:
            if len(self.buffer) >= self.min_threshold:
                self.playback_started = True
                return self.empty_buffer()
        else:
            return chunk
        return None

    def empty_buffer(self):
        data = b"".join(self.buffer)
        self.buffer = []
        return data

The second lever costs only a few lines: send a test request when your application starts. This establishes the TCP connection and TLS handshake once and for all, and HTTP keep-alive lets all subsequent requests benefit from this groundwork. Without this warm-up, your first user of the day pays the price.

Measure Before Optimizing

All these estimates must be validated against your actual setup. The minimum measurement consists of timing the delay until the first fragment is received, then exiting the loop: it is this figure, not the total duration, that corresponds to what the user experiences.

import time

t0 = time.perf_counter()

with client.audio.speech.complete(
    model="voxtral-mini-tts-3-0",
    input="Latency test.",
    voice_id=voice_id,
    response_format="pcm",
    stream=True,
) as stream:
    for event in stream:
        if event.event == "speech.audio.delta":
            t1 = time.perf_counter()
            print(f"First chunk received in {(t1 - t0)*1000:.0f} ms")
            break

Run this measurement from your production server, not your workstation: the network difference is often enough to explain the gap between a convincing demo and a disappointing deployment.

Key Takeaways

• Total latency includes network, model (~90 ms), encoding, and client buffer
• PCM format offers the lowest latency because there is no encoding
• Streaming reduces perceived latency by a factor of 3 to 4 compared to batch
• Use a voice_id rather than ref_audio to save 50-100 ms
• Generate sentence by sentence to minimize time to first sound