Real-time function calling

Update November 2, 2026 — Audio models. Several models in this course are deprecated: tts-1, tts-1-hd, and gpt-4o-mini-tts will stop working on January 6, 2027 (the indicated replacement is gpt-realtime-2.1-mini, which the OpenAI TTS guide states is not supported by /v1/audio/speech: speech synthesis migration moves to the Realtime API); gpt-realtime, gpt-4o-realtime, and legacy audio models on January 20, 2027 (gpt-realtime-2.1, gpt-realtime-2.1-mini, gpt-audio-1.5); whisper-1, gpt-4o-transcribe, and /v1/audio/translations on February 26, 2027 (gpt-transcribe, gpt-live-transcribe, v1/realtime/translations). For full-duplex voice, GPT-Live 1 has been available in the API since September 10, 2026.

Real-time function calling

Function calling in the Realtime API lets your voice agent perform concrete actions during a conversation: querying a database, placing an order, or checking a status. The mechanics are similar to the Responses API, but adapted for the WebSocket event stream.

Declaring tools in the session

You declare tools in session.update, exactly like instructions and voice. Each tool is a JSON schema describing its parameters:

tools = [
    {
        "type": "function",
        "name": "get_weather",
        "description": "Get the current weather for a given city",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "Name of the city"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit"
                }
            },
            "required": ["city"]
        }
    },
    {
        "type": "function",
        "name": "book_appointment",
        "description": "Book an appointment",
        "parameters": {
            "type": "object",
            "properties": {
                "date": {"type": "string", "description": "Date in YYYY-MM-DD format"},
                "time": {"type": "string", "description": "Time in HH:MM format"},
                "service": {"type": "string", "description": "Type of service"}
            },
            "required": ["date", "time", "service"]
        }
    }
]

await ws.send(json.dumps({
    "type": "session.update",
    "session": {
        "type": "realtime",
        "output_modalities": ["audio"],
        "instructions": "You are a voice assistant. Use the available tools.",
        "audio": {"output": {"voice": "marin"}},
        "tools": tools,
        "tool_choice": "auto"
    }
}))

The function call flow

When the model decides to call a tool, the event sequence is as follows:

Implementing the function handler

Here is a complete example of handling function calls:

import json

# Registry of available functions
async def get_weather(city: str, unit: str = "celsius") -> dict:
    # Call a real weather API
    return {"city": city, "temperature": 22, "unit": unit, "condition": "sunny"}

async def book_appointment(date: str, time: str, service: str) -> dict:
    # Booking logic
    return {"confirmed": True, "date": date, "time": time, "reference": "RDV-2026-0042"}

FUNCTION_REGISTRY = {
    "get_weather": get_weather,
    "book_appointment": book_appointment,
}

async def handle_function_call(ws, event):
    """Processes a function call and returns the result."""
    response = event["response"]

    for output in response.get("output", []):
        if output["type"] != "function_call":
            continue

        func_name = output["name"]
        call_id = output["call_id"]
        arguments = json.loads(output["arguments"])

        print(f"Calling function: {func_name}({arguments})")

        # Execute the function
        func = FUNCTION_REGISTRY.get(func_name)
        if func:
            result = await func(**arguments)
        else:
            result = {"error": f"Unknown function: {func_name}"}

        # Send the result back to the model
        await ws.send(json.dumps({
            "type": "conversation.item.create",
            "item": {
                "type": "function_call_output",
                "call_id": call_id,
                "output": json.dumps(result)
            }
        }))

    # Ask the model to continue with the result
    await ws.send(json.dumps({
        "type": "response.create"
    }))

Integrating into the event loop

The function handler integrates into your main loop:

async def event_loop(ws):
    """Main event processing loop."""
    async for raw_message in ws:
        event = json.loads(raw_message)
        event_type = event["type"]

        if event_type == "response.done":
            response = event["response"]
            # Check if the model called any functions
            has_function_calls = any(
                o["type"] == "function_call"
                for o in response.get("output", [])
            )
            if has_function_calls:
                await handle_function_call(ws, event)

        elif event_type == "response.output_audio.delta":
            # Play response audio
            audio_bytes = base64.b64decode(event["delta"])
            player.write(audio_bytes)

        elif event_type == "error":
            print(f"Error: {event['error']['message']}")

Multiple function calls

The model may decide to call multiple functions in a single response. The output field of response.done can contain multiple entries of type function_call. Process all of them before calling response.create.

Key takeaways

• Tools are declared in session.update using a standard JSON schema
• Arguments arrive via streaming through response.function_call_arguments.delta
• After execution, send the result back via conversation.item.create with the call_id
• Call response.create so the model can formulate its voice response with the result
• Handle multiple calls: the model can call several functions at once