Sessions, events, and lifecycle

Updated November 2, 2026 — Audio models. Several models in this course are deprecated: tts-1, tts-1-hd, and gpt-4o-mini-tts will stop working on January 6, 2027 (replacement: gpt-realtime-2.1-mini, which the OpenAI TTS guide states is not supported by /v1/audio/speech: speech synthesis migration moves to the Realtime API); gpt-realtime, gpt-4o-realtime, and legacy audio models on January 20, 2027 (gpt-realtime-2.1, gpt-realtime-2.1-mini, gpt-audio-1.5); whisper-1, gpt-4o-transcribe, and /v1/audio/translations on February 26, 2027 (gpt-transcribe, gpt-live-transcribe, v1/realtime/translations). For full-duplex voice, GPT-Live 1 has been available in the API since September 10, 2026.

Corrected November 3, 2026. Input transcription was using gpt-live-transcribe, which only works in a transcription session and supports neither server_vad nor semantic_vad. In a conversation session like this one, where VAD validates each turn, the documentation specifies gpt-transcribe, which transcribes validated turns and uses previous turns as context. Sources: Realtime transcription · GPT-Transcribe · GPT-Live-Transcribe.

Every WebSocket connection to the Realtime API creates a session. This session is the fundamental unit of management: it encapsulates the model configuration, conversation history, available tools, and the current state of the interaction. Understanding its lifecycle is essential for building robust voice agents.

The lifecycle of a session

As soon as the WebSocket connection is established, the server emits a session.created event. The session goes through several phases:

Connection → session.created

The server confirms the session with a unique ID and default configuration.

Configuration → session.update

You send session.update to define instructions, voice, tools, and modalities.

Active conversation

Exchange of audio and text events between client and server.

Closure → WebSocket disconnect

The session is destroyed when the connection closes. No state is retained on the server.

Configuring a session

After session.created, send a session.update event to configure the model’s behavior. Transcribing what the user says (audio.input.transcription) is optional: without it, you receive the voice but not the text. languages specifies the expected languages and improves recognition:

import asyncio
import websockets
import json

async def configure_session():
    headers = {
        "Authorization": f"Bearer {OPENAI_API_KEY}"
    }

    async with websockets.connect(URL, additional_headers=headers) as ws:
        # Wait for session.created
        event = json.loads(await ws.recv())
        session_id = event["session"]["id"]
        print(f"Session created: {session_id}")

        # Configure the session
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "type": "realtime",
                "output_modalities": ["audio"],
                "instructions": "You are a French-speaking voice assistant. "
                                "Respond concisely and naturally.",
                "audio": {
                    "input": {
                        "format": {"type": "audio/pcm", "rate": 24000},
                        "transcription": {"model": "gpt-transcribe", "languages": ["fr"]},
                        "turn_detection": {
                            "type": "server_vad",
                            "threshold": 0.5,
                            "prefix_padding_ms": 300,
                            "silence_duration_ms": 500
                        }
                    },
                    "output": {
                        "format": {"type": "audio/pcm", "rate": 24000},
                        "voice": "marin"
                    }
                },
                "tools": []
            }
        }))

        # Wait for confirmation (other events may arrive first)
        while True:
            event = json.loads(await ws.recv())
            if event["type"] == "session.updated":
                print("Session updated")
                break
            if event["type"] == "error":
                raise RuntimeError(event["error"]["message"])

Key session parameters

Parameter	Description	Recommended value
output_modalities	Output modalities	["audio"] (or ["text"])
audio.output.voice	Synthesis voice	marin, cedar, alloy, echo, etc.
audio.input.turn_detection	End-of-speech detection	server_vad with threshold 0.5
audio.input.format	Incoming audio format	{"type": "audio/pcm", "rate": 24000}

The event system

The Realtime API is entirely event-driven. Every exchanged message is a JSON object with a type field that determines its nature. Events fall into two categories:

Client events (you → server)

# The main events you send
client_events = {
    "session.update":              "Modify configuration",
    "input_audio_buffer.append":   "Send an audio chunk",
    "input_audio_buffer.commit":   "Commit the audio buffer",
    "input_audio_buffer.clear":    "Clear the audio buffer",
    "conversation.item.create":    "Add a message manually",
    "conversation.item.truncate":  "Truncate a response",
    "conversation.item.delete":    "Delete a message",
    "response.create":             "Trigger a response",
    "response.cancel":             "Cancel the current response",
}

Server events (server → you)

# The main events you receive
server_events = {
    "session.created":             "Session initialized",
    "session.updated":             "Configuration applied",
    "input_audio_buffer.speech_started":  "Speech detected",
    "input_audio_buffer.speech_stopped":  "Silence detected",
    "conversation.item.created":   "New message in history",
    "response.created":            "Generation started",
    "response.output_audio.delta":        "Response audio chunk",
    "response.output_audio_transcript.delta": "Transcription chunk",
    "response.done":               "Response finished",
    "error":                       "Error",
}

Error handling

Errors are emitted via an event of type error. You must always intercept them:

async def handle_events(ws):
    async for raw_message in ws:
        event = json.loads(raw_message)
        event_type = event["type"]

        if event_type == "error":
            error = event["error"]
            print(f"Error [{error['type']}]: {error['message']}")

            if error["type"] == "invalid_request_error":
                # Configuration error, fix and retry
                pass
            elif error["type"] == "server_error":
                # Issue on OpenAI's side, reconnect
                pass

        elif event_type == "session.updated":
            print("Configuration applied successfully")

        elif event_type == "response.done":
            usage = event["response"].get("usage", {})
            print(f"Tokens used: {usage}")

Key takeaways

A session is created automatically upon WebSocket connection and destroyed upon disconnection
session.update configures the session after session.created; wait for session.updated before relying on the new configuration
To transcribe input in a conversation session, use gpt-transcribe; gpt-live-transcribe is reserved for transcription sessions
The protocol is purely event-driven: every message has a type that determines its handling
Client events drive the interaction; server events report results
Error handling is critical: always listen for error events