Telephony: SIP, LiveKit, and G.711 — Voice Agent API: real-time conversations

Updated November 3, 2026, based on the xAI SIP Phone Calls guide, Speech to Speech page, and Phone Voice Agent cookbook example. xAI can answer calls on your number itself (direct SIP, signed webhook, transfer, hang-up, DTMF keys), alongside a bridge through your own server. Sources: SIP Phone Calls · Speech to Speech · Phone Voice Agent.

Connect the Voice Agent API to the phone network

The Voice Agent API can take phone calls: your customers dial an ordinary number and talk with your Grok agent. Three paths get you there, from the most integrated to the most flexible.

Path 1: direct SIP with xAI

In this mode, xAI answers the call itself. The call lands on xAI’s SIP infrastructure, which notifies you by webhook; your app joins the call over WebSocket and drives the agent.

Register the number. Declare a “Direct SIP” number with xAI, along with the webhook address that will receive incoming calls. For a number you own, use origin: "byo_trunk": xAI doesn’t provide numbers through the API. The response contains a signing secret, returned only once, so store it in your secrets vault.
Route your carrier. Configure your carrier or PBX to send calls to sip:{number}@sip.voice.x.ai. xAI accepts SIP signaling over TLS on port 5061, or TCP or UDP on port 5060; prefer ;transport=tls when your carrier supports it.
Receive the webhook. For each call, xAI sends a signed realtime.call.incoming webhook. Verify the webhook-id, webhook-timestamp, and webhook-signature headers with the signing secret, then read data.call_id.
Join the call. Open wss://api.x.ai/v1/realtime?call_id={call_id} with your API key (ephemeral tokens aren’t accepted for a SIP session), send session.update, then response.create when the agent should speak.

import json
import os

import websockets


async def join_call(call_id):
    async with websockets.connect(
        f"wss://api.x.ai/v1/realtime?call_id={call_id}",
        additional_headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
    ) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "voice": "rex",
                "instructions": "You are the customer service agent. Answer in English, briefly.",
                "turn_detection": {"type": "server_vad"},
            },
        }))
        await ws.send(json.dumps({"type": "response.create"}))  # the agent greets the caller

        async for message in ws:
            event = json.loads(message)
            # same handling as for a regular WebSocket session

Once connected, the session behaves like any other voice session: xAI relays the caller’s audio and plays the agent back to them, without your server carrying any sound.

Transfer or hang up

Two REST calls control the line. POST /v1/realtime/calls/{call_id}/refer, with a target_uri in tel: or sip: form, transfers the caller; the request waits for the outcome of the transfer. A 200 with an empty body means the destination answered; a 502 carries the SIP rejection code, and a 504 signals a timeout. After a failure, the session stays open and the agent can keep talking: it’s up to you to read the response and, if needed, have the agent explain the hiccup. POST /v1/realtime/calls/{call_id}/hangup ends the call.

Keypad input (DTMF)

On a SIP session, the keys the caller presses are buffered and then passed to the model as text as soon as the caller presses #, after 2.5 seconds without a new key, or as soon as they start speaking. Your client receives one input_audio_buffer.dtmf_event_received event per key, so you can keep a record. To let the agent press keys itself, for example to navigate another service’s phone menu, add {"type": "function", "name": "send_dtmf"} to its tools, with no parameters: the API provides the schema and sends the tones, and you must not answer these function calls. These events exist only on a SIP session.

Documented carriers

The xAI guide walks through the setup for four carriers (Twilio, with Elastic SIP Trunking and call transfer enabled on the trunk if you transfer calls; Bandwidth; Telnyx; and Plivo) and for any carrier, contact center, or PBX that can open a SIP route to your number’s address. On the codec side, the Telnyx example asks you to enable at least G.711 μ-law, G.711 A-law, or G.722.

Path 2: a bridge through your own server

If you want to keep control of the audio stream, your server receives the call audio and relays it to a regular WebSocket session. That’s the architecture of the xAI cookbook’s Phone Voice Agent example: Twilio streams the call audio in μ-law through its Media Streams, and the server passes it to the API as is, with no conversion, in both directions.

# Phone call → carrier → your server → xAI WebSocket
import base64
import json
import os

import websockets


async def relay_call(call_audio_stream):
    ws = await websockets.connect(
        "wss://api.x.ai/v1/realtime?model=grok-voice-latest",
        additional_headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
    )

    # G.711 μ-law at 8000 Hz, the North American phone network format
    await ws.send(json.dumps({
        "type": "session.update",
        "session": {
            "voice": "rex",
            "instructions": "Customer service agent.",
            "turn_detection": {"type": "server_vad"},
            "audio": {
                "input": {"format": {"type": "audio/pcmu", "rate": 8000}},
                "output": {"format": {"type": "audio/pcmu", "rate": 8000}},
            },
        },
    }))

    async for chunk in call_audio_stream:
        await ws.send(json.dumps({
            "type": "input_audio_buffer.append",
            "audio": base64.b64encode(chunk).decode(),
        }))

The API natively accepts both G.711 variants, which avoids any transcoding: audio/pcmu (μ-law, the North American and Japanese standard) and audio/pcma (A-law, the standard in Europe and most other countries), both at 8000 Hz.

{
  "audio": {
    "input": {"format": {"type": "audio/pcma", "rate": 8000}},
    "output": {"format": {"type": "audio/pcma", "rate": 8000}}
  }
}

DTMF keys and the REST transfer and hang-up calls belong to the direct SIP path: in a bridge, your carrier handles them.

Path 3: a real-time framework

The xAI documentation lists two ready-made integrations: LiveKit Agents and Pipecat. They handle real-time transport (WebRTC, rooms, call routing) and plug the Grok voice agent in at the right place, which spares a team that doesn’t want to build a telephony layer from doing so.

In production

Handle silences. On the phone, a caller who goes quiet leaves a dead line. turn_detection.idle_timeout_ms makes the agent check in with the caller after a silence of the length you choose (lesson 6).

Pick up a conversation on a new call. Session resumption also works over SIP: enable resumption.enabled on the first call and keep its call_id. When the customer calls back, open wss://api.x.ai/v1/realtime?call_id={new_call_id}&conversation_id={previous_call_id} and send the same session.update again. The history expires after 30 minutes of inactivity.

Follow the rules. Tell callers they’re talking to an AI and, if you record calls, say so at the start. For health data, xAI says a BAA (Business Associate Agreement) can be signed after a questionnaire.

Measure your latency. The delay callers perceive adds up the phone network, your bridge if you have one, and response generation. Measure it end to end on real calls, and keep the functions the agent calls during the conversation fast.

Key takeaways

With direct SIP, xAI answers your number, notifies you with a signed realtime.call.incoming webhook, and you join the call through wss://api.x.ai/v1/realtime?call_id=…
refer transfers the call (the outcome comes back in the HTTP response), and hangup ends it
DTMF keys reach the model and are logged through input_audio_buffer.dtmf_event_received; the send_dtmf tool lets the agent press keys
In a bridge through your server, native G.711 (audio/pcmu, audio/pcma) avoids transcoding
LiveKit Agents and Pipecat are the integrations xAI lists