SIP telephony and LiveKit integration

Updated November 3, 2026, based on the xAI SIP Phone Calls guide, Voice page, and Phone Voice Agent cookbook example. xAI can answer calls on your number over SIP itself, with call transfer, hang-up, and DTMF keypad support. Sources: SIP Phone Calls · Voice · Phone Voice Agent.

Connect Grok to the phone network

The Voice Agent isn’t limited to web and mobile apps. Your assistant can answer an ordinary phone number: customers dial it from a landline or a cell phone, and the Grok agent picks up. Callers don’t need to install anything, which makes all the difference for older customers or areas with poor mobile internet coverage.

Three ways to connect

Direct SIP, the standard for voice over IP, is the most integrated option. You register your number with xAI, your carrier routes calls to xAI’s SIP address, and xAI notifies your app with a signed webhook for every incoming call; your app then joins the call over WebSocket. The xAI guide walks through the setup for Twilio, Bandwidth, Telnyx, and Plivo, and for any carrier or PBX that can open a SIP route to that number.

A bridge through your own server suits teams that want to keep control of the audio stream. Your server receives the call audio (for example, Twilio Media Streams, as in the xAI cookbook’s Phone Voice Agent example) and relays it to a regular WebSocket session.

Real-time frameworks, finally: the xAI documentation lists LiveKit and Pipecat among existing integrations. They handle real-time transport and call routing, which makes them the shortest path for a team that doesn’t want to build its own telephony layer.

Configure audio for the phone

With direct SIP, you don’t have to worry about it: xAI relays the caller’s audio into the session and plays the agent’s voice back to them. In a bridge, however, the format matters. The phone network runs on narrowband audio with its own codecs, and the Voice Agent accepts them natively, which avoids any transcoding.

G.711 μ-law, declared as audio/pcmu, runs at 8000 Hz; it’s the standard in North American and Japanese telephony. G.711 A-law, declared as audio/pcma, has the same characteristics and dominates European and international telephony. So you choose based on where your carrier operates, not on personal preference.

{
  "type": "session.update",
  "session": {
    "voice": "ara",
    "instructions": "You are the phone assistant for Miller & Sons. Answer professionally and concisely.",
    "audio": {
      "input": { "format": { "type": "audio/pcma", "rate": 8000 } },
      "output": { "format": { "type": "audio/pcma", "rate": 8000 } }
    },
    "turn_detection": { "type": "server_vad", "idle_timeout_ms": 8000 }
  }
}

Turn detection is still handled by the server, with no threshold to tune. The setting that matters on the phone is idle_timeout_ms: if the caller goes quiet for that long after a response, the agent checks in rather than leaving the line silent.

What direct SIP adds

Three features exist only on the direct SIP path. First, transfer: POST /v1/realtime/calls/{call_id}/refer sends the caller to a phone number (tel:) or a SIP address (sip:), and the HTTP response tells you whether the destination answered; if it fails, the caller stays on the line with the agent. Second, hang-up, with POST /v1/realtime/calls/{call_id}/hangup. Third, keypad input: the digits the caller presses reach the model as text (after #, after 2.5 seconds without a new key, or as soon as the caller speaks), and the send_dtmf tool lets the agent press keys itself, for example to navigate another service’s phone menu.

What you can build with it

An automated switchboard comes first. The agent greets callers on behalf of the company, figures out what they need as the conversation unfolds, then either transfers them to the right department with refer or handles simple requests itself. Unlike a press-1-for-sales phone menu, callers describe their problem in their own words instead of navigating a tree, and if they prefer the keypad, DTMF is still there.

A technical support hotline goes further. The agent diagnoses the issue through a series of questions, gives step-by-step instructions, creates a ticket through function calling, and transfers to a human technician when the case is beyond its scope. Session resumption even lets the conversation continue if the customer calls back within 30 minutes, as long as you’ve enabled resumption and can retrieve the ID of the first call.

Phone-specific requirements

Latency trumps everything else. A noticeable lag pushes callers to talk over the agent, and the conversation falls apart. Three levers keep it in check: avoid transcoding, host your app as close to your carrier as possible, and keep the functions the agent calls fast, since a single query to a slow CRM can blow the whole latency budget. Measure end-to-end delay on real calls rather than relying on a benchmark figure.

Next comes compliance, which isn’t negotiable. Tell callers they’re talking to an AI and, if the conversation is recorded, say so at the start. Treat transcripts and recordings as personal data under the GDPR, with defined retention periods and a deletion process that actually works.

Finally, plan for failure, because it will happen. A fallback message must take over if the Voice Agent is unavailable, or your customers will hit a dead line. A transfer to a human should kick in after two or three consecutive misunderstandings. And logging transcripts gives you the material to spot recurring failures and refine your instructions.

Key takeaways

• Three options: direct SIP with xAI, a bridge through your server, or a framework such as LiveKit or Pipecat
• With direct SIP, xAI answers the call, notifies your app with a signed webhook, and supports transfer, hang-up, and DTMF keys
• In a bridge, G.711 codecs (audio/pcmu, audio/pcma) at 8000 Hz avoid transcoding
• Turn detection has no threshold to tune; idle_timeout_ms prompts a silent caller
• Compliance (informing callers, recording notices, GDPR) is mandatory in telephony