Real-time audio streaming

Update November 2, 2026 — Audio models. Several models in this course are deprecated: tts-1, tts-1-hd, and gpt-4o-mini-tts will stop working on January 6, 2027 (successor: gpt-realtime-2.1-mini, which the OpenAI TTS guide states is not supported by /v1/audio/speech: voice synthesis migration moves to the Realtime API); gpt-realtime, gpt-4o-realtime, and legacy audio models on January 20, 2027 (gpt-realtime-2.1, gpt-realtime-2.1-mini, gpt-audio-1.5); whisper-1, gpt-4o-transcribe, and /v1/audio/translations on February 26, 2027 (gpt-transcribe, gpt-live-transcribe, v1/realtime/translations). For full-duplex voice, GPT-Live 1 has been available in the API since September 10, 2026. Updated November 3, 2026: The LLM → TTS pipeline previously processed any event carrying a delta field, including function call arguments or reasoning summaries; it now filters strictly for response.output_text.delta, the actual response text.

In a voice agent, waiting for the entire audio file to generate before starting playback introduces unacceptable latency. TTS streaming lets you begin playback as soon as the first bytes arrive, cutting the time to first response down to a few hundred milliseconds.

Streaming with the TTS API

The TTS API supports streaming natively. Instead of waiting for the complete file, you receive audio in chunks:

from openai import OpenAI
import pyaudio

client = OpenAI()

def stream_tts_to_speaker(text: str, voice: str = "alloy"):
    """Generate and play audio in streaming mode."""
    # Initialize the audio player
    pa = pyaudio.PyAudio()
    player = pa.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=24000,
        output=True
    )

    try:
        # Create the streaming response
        with client.audio.speech.with_streaming_response.create(
            model="tts-1",
            voice=voice,
            input=text,
            response_format="pcm"  # Raw PCM for direct playback
        ) as response:
            # Read and play chunk by chunk
            for chunk in response.iter_bytes(chunk_size=4096):
                player.write(chunk)

    finally:
        player.stop_stream()
        player.close()
        pa.terminate()

stream_tts_to_speaker("Hello, here is a real-time streaming response.")

The pcm format is ideal for streaming because it requires no decoding: the raw bytes are ready to play immediately.

Measuring latency

Time to First Byte (TTFB) is the key metric for streaming performance:

import time

def measure_tts_latency(text: str, voice: str = "alloy") -> dict:
    """Measure TTS streaming latencies."""
    start_time = time.time()
    first_byte_time = None
    total_bytes = 0

    with client.audio.speech.with_streaming_response.create(
        model="tts-1",
        voice=voice,
        input=text,
        response_format="pcm"
    ) as response:
        for chunk in response.iter_bytes(chunk_size=4096):
            if first_byte_time is None:
                first_byte_time = time.time()
            total_bytes += len(chunk)

    end_time = time.time()

    return {
        "ttfb_ms": round((first_byte_time - start_time) * 1000),
        "total_ms": round((end_time - start_time) * 1000),
        "total_bytes": total_bytes,
        "audio_duration_s": round(total_bytes / (24000 * 2), 2)
    }

metrics = measure_tts_latency("Hello, how are you?")
print(f"TTFB: {metrics['ttfb_ms']}ms, Total: {metrics['total_ms']}ms")

Asynchronous streaming

For server-side applications, use the asynchronous client:

from openai import AsyncOpenAI
import asyncio

async_client = AsyncOpenAI()

async def stream_tts_async(text: str, voice: str = "alloy"):
    """Asynchronous TTS streaming for servers."""
    chunks = []

    async with async_client.audio.speech.with_streaming_response.create(
        model="tts-1",
        voice=voice,
        input=text,
        response_format="pcm"
    ) as response:
        async for chunk in response.iter_bytes(chunk_size=4096):
            chunks.append(chunk)
            # Here, you would send the chunk to the client via WebSocket
            yield chunk

# Usage in a FastAPI server
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

@app.get("/tts/stream")
async def tts_stream(text: str, voice: str = "alloy"):
    return StreamingResponse(
        stream_tts_async(text, voice),
        media_type="audio/pcm"
    )

Pre-fill buffer

To ensure smooth playback without gaps, accumulate a small buffer before starting:

async def buffered_tts_playback(
    text: str,
    voice: str = "alloy",
    buffer_size: int = 3  # Number of chunks to pre-load
):
    """TTS playback with a pre-fill buffer."""
    pa = pyaudio.PyAudio()
    player = pa.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=24000,
        output=True
    )

    buffer = []
    playback_started = False

    try:
        async with async_client.audio.speech.with_streaming_response.create(
            model="tts-1",
            voice=voice,
            input=text,
            response_format="pcm"
        ) as response:
            async for chunk in response.iter_bytes(chunk_size=4096):
                buffer.append(chunk)

                if not playback_started and len(buffer) >= buffer_size:
                    playback_started = True
                    # Flush the initial buffer
                    for buffered_chunk in buffer:
                        player.write(buffered_chunk)
                    buffer.clear()
                elif playback_started:
                    player.write(chunk)

        # Play remaining chunks
        for remaining in buffer:
            player.write(remaining)

    finally:
        player.stop_stream()
        player.close()
        pa.terminate()

Integrating with an LLM + TTS pipeline

In a non-Realtime voice agent, you can combine LLM streaming with TTS streaming:

async def llm_to_tts_pipeline(user_message: str):
    """Pipeline: Streaming LLM → Real-time streaming TTS."""
    # Phase 1: Accumulate LLM response by sentence
    sentence_buffer = ""

    stream = client.responses.create(
        model="gpt-5.6-luna",
        input=[{"role": "user", "content": user_message}],
        stream=True
    )

    for event in stream:
        # Only response text should be spoken
        if event.type == "response.output_text.delta":
            sentence_buffer += event.delta

            # Detect end of sentence
            if sentence_buffer.rstrip().endswith((".", "!", "?")):
                sentence = sentence_buffer.strip()
                sentence_buffer = ""

                # Phase 2: Stream this sentence to TTS
                stream_tts_to_speaker(sentence)

    # Process remaining buffer
    if sentence_buffer.strip():
        stream_tts_to_speaker(sentence_buffer.strip())

Key takeaways

TTS streaming cuts TTFB to a few hundred milliseconds by starting playback before generation finishes
The pcm format is optimal for streaming: no decoding required, direct playback
A pre-fill buffer of 2–3 chunks prevents gaps at the start of playback
The asynchronous client is essential for servers handling multiple simultaneous connections
Combining streaming LLM and streaming TTS enables a responsive voice agent without the Realtime API