Introduction to Real-Time Transcription

Course updated November 3, 2026 for Voxtral 3, based on Mistral’s Migrate from Voxtral 2 guide: Voxtral 2 models (voxtral-mini-2602, voxtral-mini-transcribe-realtime-2602, voxtral-mini-tts-2603), deprecated on October 31, 2026 and retired on April 30, 2027, are replaced in examples by voxtral-mini-transcribe-3, voxtral-mini-transcribe-realtime-3, and voxtral-mini-tts-3-0. Key changes: timestamps are segment-only (word returns a 400 error), diarization is available in real time, real-time session settings are locked after the first audio chunk, and context_bias inputs are free-form (phrases with spaces). Sources: Migrate from Voxtral 2 · Speech to Text.

A New Paradigm: Audio Streaming

So far, you have worked in offline mode: you send a complete audio file, wait for the model to process it, and then use the result. This workflow suits existing recordings, such as archived meetings, podcasts, or interviews. It fails when audio arrives continuously: no one will wait until the end of a two-hour conference to see the first subtitles, and a voice assistant that only understands a question after a long silence seems broken.

Voxtral Realtime meets this need with a streaming architecture. Audio is transcribed on the fly, word by word, and the delay between sound and text becomes a setting you control, dropping as low as 200 milliseconds. The shift is as much mental as technical: you no longer think in isolated requests but in streams, and your program must be ready to receive text at any moment.

What the Realtime Model Offers

The model is invoked under the identifier voxtral-mini-transcribe-realtime-3 and differs fundamentally from its offline counterpart. It receives audio chunks and emits text fragments continuously, never waiting for the recording to end. Its latency is controlled by the target_streaming_delay_ms parameter, which you will explore in detail later in this course. It covers 45 languages and dialects, and the language can be fixed or detected. Its Generation 2 predecessor, voxtral-mini-transcribe-realtime-2602 (4 billion parameters), is published as open weights under the Apache 2.0 license on the Hugging Face Hub for fully sovereign installation on your own servers.

With Voxtral 3, real-time performance has largely closed the gap with offline mode. Diarization is available—though not in the same session as turn-end detection—as are context biasing and turn-end detection. Two differences remain: segments carry the speaker label but no start or end time, and all session settings (language, vocabulary, diarization, delay) must be sent before the first audio chunk, after which they are locked. If your use case requires precise timestamps, you need offline mode, possibly as a complement to real time.

Installation

The Mistral SDK does not load streaming dependencies by default. You must explicitly request the realtime extra, which adds support for WebSocket connections and asynchronous audio processing:

pip install "mistralai[realtime]"

The Required Audio Format

Unlike offline mode, which accepts a wide range of containers, Voxtral Realtime expects a raw stream in a single, non-negotiable format:

• Encoding: 16-bit signed little-endian PCM (pcm_s16le)
• Sample rate: 16,000 Hz (16 kHz)
• Channels: Mono (1 channel)

This constraint is not a blocker: this format is the most standard available, and all capture tools produce it natively, whether PyAudio, FFmpeg, or PulseAudio. You declare it once to the client, which uses it to interpret the bytes you transmit.

from mistralai.client.models import AudioFormat

# Define the audio format
audio_format = AudioFormat(
    encoding="pcm_s16le",
    sample_rate=16000
)

The Two Stream Events

When you iterate over the transcription, the SDK returns two types of objects. TranscriptionStreamTextDelta is emitted for each new transcribed fragment and carries the partial text you can display immediately; this is what powers the user’s screen. TranscriptionStreamDone arrives exactly once, when the audio stream ends, whether at the end of a file or when the microphone closes; it signals that nothing more is coming and that you can consolidate the result. Your entire logic therefore fits in an asynchronous loop that distinguishes these two cases:

from mistralai.client.models import (
    TranscriptionStreamTextDelta,
    TranscriptionStreamDone
)

async for event in flux_transcription:
    if isinstance(event, TranscriptionStreamTextDelta):
        # New text available
        print(event.text, end="", flush=True)
    elif isinstance(event, TranscriptionStreamDone):
        # Transcription finished
        print("\n[Finished]")

First Complete Example

To get comfortable with the mechanism without relying on a microphone, the simplest approach is to simulate a stream from a file already converted to PCM. The lire_fichier_pcm function is an asynchronous generator: it reads the file in blocks and yields them one by one, exactly as a sound card would. The rest of the program is what you will reuse later without modification.

import asyncio
import os
from mistralai.client import Mistral
from mistralai.client.models import (
    AudioFormat,
    TranscriptionStreamTextDelta,
    TranscriptionStreamDone
)

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
audio_format = AudioFormat(encoding="pcm_s16le", sample_rate=16000)

async def lire_fichier_pcm(chemin: str, taille_chunk: int = 32000):
    """Reads a PCM file in chunks to simulate a stream."""
    with open(chemin, "rb") as f:
        while True:
            chunk = f.read(taille_chunk)
            if not chunk:
                break
            yield chunk

async def main():
    flux_audio = lire_fichier_pcm("audio.pcm")

    async for event in client.audio.realtime.transcribe_stream(
        audio_stream=flux_audio,
        model="voxtral-mini-transcribe-realtime-3",
        audio_format=audio_format,
    ):
        if isinstance(event, TranscriptionStreamTextDelta):
            print(event.text, end="", flush=True)
        elif isinstance(event, TranscriptionStreamDone):
            print("\n\nTranscription finished.")

asyncio.run(main())

If your source file is an MP3, it cannot be sent as is: convert it first with FFmpeg, enforcing the three characteristics required by the model.

ffmpeg -i audio.mp3 -f s16le -acodec pcm_s16le -ar 16000 -ac 1 audio.pcm

When to Switch to Real Time

The selection criterion is simple: real time is justified when someone is waiting for the text while the sound is being produced. This applies to live subtitling for a conference, webinar, or online course, where the audience reads as they listen; to a voice assistant, which must understand the user while they speak to respond without awkward pauses; to dictation, where speech becomes text before the speaker’s eyes; and finally to conference transcription, when the text must appear as it is spoken. Everywhere else, offline mode remains richer and simpler to implement.

The following lessons will show you how to capture the microphone, tune latency, and build these use cases end to end.

Key Takeaways

• Voxtral Mini Transcribe Realtime 3 transcribes audio in streaming mode with an adjustable delay from 80 ms to 1.2 s (or 2.4 s)
• Required audio format: 16-bit PCM, 16 kHz, mono
• Two event types: TextDelta (partial text) and Done (end)
• Install with pip install "mistralai[realtime]"
• Asynchronous programming required (async/await)
• Diarization (excluding turn-end detection) and context biasing available in real time, without segment timestamps; settings locked after the first audio chunk
• Generation 2 of the realtime model is open weights Apache 2.0, for sovereign deployment