Next Steps: Beyond Transcription

Course updated November 3, 2026 for Voxtral 3, based on Mistral’s Migrate from Voxtral 2 guide: Voxtral 2 models (voxtral-mini-2602, voxtral-mini-transcribe-realtime-2602, voxtral-mini-tts-2603), deprecated on October 31, 2026 and retired on April 30, 2027, are replaced in examples by voxtral-mini-transcribe-3, voxtral-mini-transcribe-realtime-3, and voxtral-mini-tts-3-0. Key changes: timestamps are segment-level only (word returns a 400 error), real-time diarization is supported, real-time session settings are locked after the first audio chunk, and context_bias inputs are free-form (phrases with spaces). Sources: Migrate from Voxtral 2 · Speech to Text.

What You’ve Learned

You’ve completed the journey through both Voxtral transcription modes. In offline mode, you transcribe audio files with diarization, timestamps, and context biasing using Voxtral Mini Transcribe 3. In real-time mode, you transcribe continuous streams with configurable latency via Voxtral Realtime, capture microphone input, and leverage the dual delay pattern. For each capability, you now have complete Python code and the instincts needed to move to production.

Transcription is only half the story, though. Mistral AI’s audio ecosystem also handles the reverse path, and combining both directions is what enables truly conversational applications.

Voxtral TTS: Giving Voice to Text

Voxtral TTS is the exact counterpart to transcription: it transforms text into natural speech. It supports zero-shot voice cloning, meaning it can reproduce a voice from just two to three seconds of audio without prior training. It also relies on the voice-as-an-instruction principle: the model follows the intonation, rhythm, and emotion of the vocal prompt you provide, so you don’t need to insert prosody tags into the text—you show the desired tone instead of describing it. Language coverage includes English, French, Spanish, Portuguese, Italian, Dutch, German, Hindi, and Arabic. It also works in streaming mode, with a model latency of around 90 milliseconds and a time-to-first-audio of about 0.8 seconds in PCM, making it suitable for live conversation.

In practice, Voxtral TTS gives your applications a voice: virtual assistants, automated narration, accessibility tools, and dubbing.

The Complete Pipeline: STT, LLM, TTS

Assembled together, these three components form a complete loop. User audio becomes text, the text is understood and processed by a language model, and the response becomes audio again.

User Audio → Voxtral STT → Text
                             ↓
                      Mistral LLM (reasoning)
                             ↓
                   Text Response → Voxtral TTS → Audio Response

The code below implements this pipeline. The first two steps are familiar: streaming transcription at 300 milliseconds to capture the question, then calling the LLM with a system prompt that enforces a spoken register. The third step remains commented out, as the synthesis API is covered in a dedicated course; the call pattern is included here to show where it fits.

import asyncio
import os
from mistralai.client import Mistral
from mistralai.client.models import (
    AudioFormat,
    TranscriptionStreamTextDelta,
    TranscriptionStreamDone
)

async def pipeline_vocal_complet(source_audio_utilisateur):
    """Complete pipeline: listen, understand, respond."""
    client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

    # Step 1: Transcribe user speech (STT)
    audio_format = AudioFormat(encoding="pcm_s16le", sample_rate=16000)
    question_parts = []

    print("Listening...", flush=True)

    async for event in client.audio.realtime.transcribe_stream(
        audio_stream=source_audio_utilisateur,
        model="voxtral-mini-transcribe-realtime-3",
        audio_format=audio_format,
        target_streaming_delay_ms=300,
    ):
        if isinstance(event, TranscriptionStreamTextDelta):
            question_parts.append(event.text)
        elif isinstance(event, TranscriptionStreamDone):
            break

    question = "".join(question_parts).strip()
    print(f"Question: {question}")

    # Step 2: Generate an intelligent response (LLM)
    reponse_llm = client.chat.complete(
        model="mistral-large-latest",
        messages=[
            {
                "role": "system",
                "content": "You are a voice assistant. "
                           "Respond concisely and naturally, "
                           "as in a spoken conversation."
            },
            {"role": "user", "content": question}
        ]
    )

    reponse_texte = reponse_llm.choices[0].message.content
    print(f"Response: {reponse_texte}")

    # Step 3: Synthesize the response into audio (TTS)
    # Note: The TTS API will be covered in a dedicated course
    # Here is the call pattern:
    #
    # audio_response = client.audio.speech.complete(
    #     model="voxtral-mini-tts-3-0",
    #     input=reponse_texte,
    #     voice_id="gb_jane_neutral",
    #     response_format="wav",
    # )
    # Audio arrives base64-encoded in audio_response.audio_data
    #
    # Play audio via a player (PyAudio, sounddevice, etc.)

    return {
        "question": question,
        "reponse": reponse_texte
    }

What This Pipeline Enables

The enterprise voice assistant is the most immediate application: it understands employee questions and responds using the internal knowledge base. Real-time transcription captures the question, the LLM queries a vector database via RAG, and TTS speaks the response aloud.

The call center agent takes the logic further by using both transcription modes simultaneously. During the call, Voxtral Realtime transcribes the conversation live; Mistral LLM analyzes customer intent; Voxtral TTS formulates the vocal response. Once the call ends, an offline pass with diarization produces a structured summary indicating who said what—precisely what real-time cannot provide.

The voice translation tool chains three models on a single stream: Voxtral STT transcribes the source speech among its 50 languages, Mistral LLM translates, and Voxtral TTS renders the output in one of its 9 target languages. Finally, automated narration turns articles, reports, or emails into audio for mobile listening, with the LLM adapting the text for oral delivery before synthesis.

A Fully Sovereign Voice Pipeline

The decisive advantage of this ecosystem is the ability to host everything on your own infrastructure. Voxtral Realtime is released as open weights under the Apache 2.0 license and deploys on your own servers. Mistral Large 3 is accessible via European cloud partners or on-premise. Voxtral TTS also supports private deployment. In other words, the entire chain—listen, understand, respond—can operate without any data leaving your infrastructure. For healthcare, defense, finance, or public administration, this isn’t just a convenience; it determines whether the project is feasible at all.

Where to Go Next

Mistral’s official documentation dedicates pages to audio and remains the reference for tracking model and API updates. The Hugging Face Hub hosts Voxtral models as open weights, along with documentation and examples. The Mistral forum gathers field reports often closer to your real-world constraints than official demos, and other Corsen Academy courses cover the rest of the Mistral AI ecosystem.

Course Recap

You now know the Voxtral models and their respective roles: Mini Transcribe V2 for offline, Realtime for streaming, and Small for audio chat. You’ve mastered offline transcription in all its dimensions—API calls, timestamps, diarization, context biasing—as well as real-time transcription with its streaming, microphone capture via PyAudio, latency control, and dual delay pattern. You’ve built complete use cases, from subtitling and meeting minutes to dictation, voice assistants, and conferences, and you know how to run all of this in production. All that’s missing is the return voice to close the conversation loop.

Key Takeaways

• Voxtral TTS closes the loop: STT (transcription) then LLM (reasoning) then TTS (synthesis)
• The complete voice pipeline enables voice assistants, call center agents, and translators
• Sovereign deployment is possible thanks to open weights models (Apache 2.0)
• Transcription is the foundational building block for voice applications
• Continue your learning with courses dedicated to TTS and advanced voice pipelines

Test Your Knowledge

Offline, real-time, production: transcription from every angle.

1. Offline or real-time: how to choose?

Answer: Offline for existing files (recorded meetings, archives) where maximum accuracy matters; real-time for live scenarios (subtitling, voice assistants) where latency is critical.

2. What do timestamps and diarization provide?

Answer: Timestamps locate each segment in time (subtitles, navigation); diarization identifies who is speaking—together, they turn audio into an actionable summary.

3. What is context biasing for?

Answer: To cue the model on expected vocabulary (proper nouns, industry terms) to improve accuracy where generic transcription fails.

4. What are the ingredients for real-time in Python?

Answer: Microphone capture (PyAudio), streaming to the API, and managing the latency/accuracy trade-off—latency parameters are tuned based on use case.

5. What do you monitor for production?

Answer: Input audio quality, costs per transcribed hour, end-to-end latency, and errors on critical vocabulary—with biasing and testing as levers.

The offline/real-time choice structures everything else—the course guide gives you the framework, your use cases drive the decision.