Latency Control

Course updated November 3, 2026 for Voxtral 3, based on Mistral’s Migrate from Voxtral 2 guide: Voxtral 2 models (voxtral-mini-2602, voxtral-mini-transcribe-realtime-2602, voxtral-mini-tts-2603), deprecated October 31, 2026 and retired April 30, 2027, are replaced in examples by voxtral-mini-transcribe-3, voxtral-mini-transcribe-realtime-3, and voxtral-mini-tts-3-0. Key changes: timestamping by segment only (word returns a 400 error), real-time diarization is possible, real-time session settings are locked after the first audio chunk, and context_bias inputs are free-form (phrases with spaces). Sources: Migrate from Voxtral 2 · Speech to Text.

The latency vs. accuracy trade-off

Real-time transcription relies on a trade-off you cannot avoid: response speed comes at the cost of accuracy. The longer the model waits before emitting a word, the more context it has to distinguish between similar acoustic hypotheses—and the longer your user waits. A phrase like “he took the sea” versus “he took the bitter” may only become clear from what follows; a rushed model will sometimes pick the wrong one.

Voxtral Realtime doesn’t hide this trade-off behind an automatic setting: it hands it to you explicitly via the target_streaming_delay_ms parameter. You decide the balance point, case by case.

The target_streaming_delay_ms parameter

This value expresses in milliseconds the target delay between receiving audio and emitting the corresponding text. With Voxtral 3, it accepts any integer from 80 to 1,200, or 2,400; any other value triggers an error event. You pass it directly to the streaming call, just like the model and audio format, and like all session settings, it locks after the first audio chunk:

import asyncio
import os
from mistralai.client import Mistral
from mistralai.client.models import (
    AudioFormat,
    TranscriptionStreamTextDelta,
    TranscriptionStreamDone
)

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
audio_format = AudioFormat(encoding="pcm_s16le", sample_rate=16000)

# Low latency: fast response, less accurate
async for event in client.audio.realtime.transcribe_stream(
    audio_stream=source_audio,
    model="voxtral-mini-transcribe-realtime-3",
    audio_format=audio_format,
    target_streaming_delay_ms=240,  # 240 ms
):
    if isinstance(event, TranscriptionStreamTextDelta):
        print(event.text, end="", flush=True)

Three value ranges cover nearly all use cases. Between 200 and 300 milliseconds, transcription is ultra-responsive: this is the setting for voice assistants, where immediacy matters most and the text may contain more corrections. Between 500 and 1,000 milliseconds, you get a good balance, suitable for live captioning and dictation. Beyond that, between 2,000 and 3,000 milliseconds, the model works with much more context and produces significantly cleaner text, at the cost of a delay the user clearly perceives.

The Dual Delay pattern

Do you really have to choose? Voxtral Realtime offers a workaround: launch two streams in parallel on the same audio, with different latencies, and thus refuse the trade-off. The fast stream, set for example to 240 milliseconds, gives immediate visual feedback that reassures the user and shows the system is listening. The slow stream, set to 2,400 milliseconds, produces a more accurate transcription that gradually replaces the provisional text.

You know this behavior even if you haven’t named it: it’s how live captions work on YouTube or Google Meet, where text appears almost instantly then corrects itself a few moments later. Implementation involves defining two coroutines identical except for the delay, then running them together with asyncio.gather.

import asyncio
import os
from mistralai.client import Mistral
from mistralai.client.models import (
    AudioFormat,
    TranscriptionStreamTextDelta,
    TranscriptionStreamDone
)

async def dual_latency_stream(source_audio_fast, source_audio_slow):
    """Runs two transcriptions in parallel with different latencies."""
    client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
    audio_format = AudioFormat(encoding="pcm_s16le", sample_rate=16000)

    results = {"fast": [], "slow": []}

    async def fast_stream():
        async for event in client.audio.realtime.transcribe_stream(
            audio_stream=source_audio_fast,
            model="voxtral-mini-transcribe-realtime-3",
            audio_format=audio_format,
            target_streaming_delay_ms=240,
        ):
            if isinstance(event, TranscriptionStreamTextDelta):
                results["fast"].append(event.text)
                # Display immediately (will be replaced by slow stream)
                print(f"\r[FAST] {''.join(results['fast'][-10:])}",
                      end="", flush=True)

    async def slow_stream():
        async for event in client.audio.realtime.transcribe_stream(
            audio_stream=source_audio_slow,
            model="voxtral-mini-transcribe-realtime-3",
            audio_format=audio_format,
            target_streaming_delay_ms=2400,
        ):
            if isinstance(event, TranscriptionStreamTextDelta):
                results["slow"].append(event.text)
                # Display corrected version
                print(f"\n[ACCURATE] {''.join(results['slow'][-10:])}",
                      flush=True)

    # Run both streams in parallel
    await asyncio.gather(fast_stream(), slow_stream())

    return {
        "fast": "".join(results["fast"]),
        "slow": "".join(results["slow"])
    }

Note that the function receives two separate audio sources, not one: an async generator can only be consumed once, so you must duplicate the stream upstream to feed both transcriptions.

Handling on-screen replacement

In a terminal, the dual output from the code above is enough to understand the mechanism, but a real interface must merge the two streams into a single, readable text. The established convention is to display provisional text in a subtle shade—gray—and confirmed text in white, with the boundary between them advancing as the slow stream catches up to the fast one. The class below implements this boundary with final_position: everything before it comes from the slow stream and won’t change, everything after comes from the fast stream and remains subject to correction.

class DualLatencyDisplay:
    """Manages display with dual latency."""

    def __init__(self):
        self.fast_text = ""
        self.final_text = ""
        self.final_position = 0

    def on_fast_text(self, text: str):
        """New text from the fast stream (provisional)."""
        self.fast_text += text
        self._display()

    def on_slow_text(self, text: str):
        """New text from the slow stream (final)."""
        self.final_text += text
        self.final_position = len(self.final_text)
        self._display()

    def _display(self):
        """Displays final text + fast text not yet confirmed."""
        provisional_text = self.fast_text[self.final_position:]
        display = self.final_text + provisional_text
        print(f"\r{display}", end="", flush=True)

This comfort comes at a cost it would be dishonest to ignore: two streams mean two connections and twice the audio volume processed. Reserve Dual Delay for interfaces where the user reads the text as it happens; for background processing, a single well-tuned stream is more than enough.

Choosing the right latency for your use case

Measuring actual latency achieved

The value you request is a target, not a guarantee: network distance, service load, and your chunk size all add to it. Before quoting a number to your users, measure it from where your application will actually run. The function below times the delay until the first fragment arrives and compares it to the requested target.

import time

async def measure_latency(source_audio, delay_ms):
    """Measures the effective latency of transcription."""
    client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
    audio_format = AudioFormat(encoding="pcm_s16le", sample_rate=16000)

    start = time.time()
    first_text = None

    async for event in client.audio.realtime.transcribe_stream(
        audio_stream=source_audio,
        model="voxtral-mini-transcribe-realtime-3",
        audio_format=audio_format,
        target_streaming_delay_ms=delay_ms,
    ):
        if isinstance(event, TranscriptionStreamTextDelta):
            if first_text is None:
                first_text = time.time()
                latency = (first_text - start) * 1000
                print(f"First text after {latency:.0f} ms "
                      f"(target: {delay_ms} ms)")
            print(event.text, end="", flush=True)
        elif isinstance(event, TranscriptionStreamDone):
            break

    print(f"\nTotal duration: {time.time() - start:.1f}s")

Run this measurement with 240, then 800, then 2,400 milliseconds on the same excerpt, and read the three transcriptions side by side. The quality gap between the first and last will tell you, better than any general recommendation, what latency your application can afford.

Key takeaways

• target_streaming_delay_ms controls the latency/accuracy trade-off
• Low values (80 to 300 ms): responsive but less accurate
• High values (1,200 ms, or 2,400 ms): accurate but with perceptible delay; only values from 80 to 1,200 and 2,400 are accepted
• The Dual Delay pattern combines two streams for both responsiveness AND accuracy
• The fast stream gives immediate feedback, the slow stream corrects the text
• Choose latency based on your use case (voice assistant vs. captioning)