WebSocket messages: text and audio
On this page
Updated November 3, 2026, based on the xAI Text to Speech page: deltas of up to 60,000 characters, cancellation (
text.clear,audio.clear), and the pronunciation table (session.update,session.updated). Source: Text to Speech.
The TTS message protocol
Once the WebSocket connection is open, communication works by exchanging JSON messages. You send text, and the server answers with base64-encoded audio. Understanding this protocol is essential to implementing TTS streaming correctly.
Client-to-server messages
Your app sends four types of messages to the server: text.delta and text.done for text, text.clear to cancel, and session.update to configure.
text.delta: send a piece of text
The text.delta message sends a fragment of text to convert. You can send the text in successive pieces:
{"type": "text.delta", "delta": "Hello, and welcome to "}{"type": "text.delta", "delta": "our presentation on speech synthesis."}Each delta can hold up to 60,000 characters, and the total text has no limit. The server starts generating audio as soon as it has enough context to produce natural pronunciation.
text.done: signal the end of the text
The text.done message tells the server you’ve finished sending text for this sequence:
{"type": "text.done"}Once it receives this message, the server finishes generating the audio and sends an audio.done message. You can then start sending new deltas for a new sequence.
text.clear: cancel the current sequence
text.clear cancels the current sequence: the server stops generating audio, discards what it had in memory, and answers with audio.clear. The connection is immediately ready for the next sequence, with no new WebSocket handshake; xAI puts the savings at about 600 ms for a distant client. It’s the interruption (“barge-in”) mechanism for when the user cuts in.
{"type": "text.clear"}session.update: fix pronunciation
session.update installs, or replaces, the connection’s replace table: each phrase found in the text is swapped for its spoken form before synthesis, without changing the text you send or the number of characters you’re billed for. Send it before the first text.delta so it covers the whole connection; an update received mid-sequence only applies from the next sequence.
{"type": "session.update", "replace": {"kubectl": "kube cuttle"}}Server-to-client messages
The server answers with five types of messages: audio.delta, audio.done, and error, detailed below, plus audio.clear, which confirms a cancellation, and session.updated, which returns the replace table in effect.
audio.delta: a piece of audio
The audio.delta message contains a base64-encoded audio fragment:
{
"type": "audio.delta",
"delta": "SUQzBAAAAAAAI1RTU0UAAAAPAAADTGF2ZjU4Ljc2..."
}Each delta is a valid piece of audio you can decode and play right away. Deltas arrive as generation progresses.
audio.done: generation finished
The audio.done message signals that all the audio for the text sequence you sent has been generated:
{
"type": "audio.done",
"trace_id": "550e8400-e29b-41d4-a716-446655440000"
}The trace_id is a unique ID for the sequence, useful for debugging and tracking.
error: an error occurred
If something goes wrong, the server sends an error message:
{
"type": "error",
"message": "Invalid text encoding"
}The full communication flow
Here’s how a typical session unfolds:
Client Server
| |
|-- text.delta ("Hello, ") ----->|
|-- text.delta ("how are ") ---->|
|-- text.delta ("you?") -------->|
|<-- audio.delta (chunk 1) -----|
|<-- audio.delta (chunk 2) -----|
|-- text.done ----------------->|
|<-- audio.delta (chunk 3) -----|
|<-- audio.done ----------------|The server can start sending audio.delta messages before you’ve even sent text.done. That’s the whole idea of streaming: generation starts as soon as possible.
Complete implementation in Python
import asyncio
import websockets
import json
import base64
async def stream_tts(api_key, text, voice="eve", language="en"):
uri = f"wss://api.x.ai/v1/tts?language={language}&voice={voice}&codec=mp3"
headers = {"Authorization": f"Bearer {api_key}"}
audio_chunks = []
async with websockets.connect(uri, additional_headers=headers) as ws:
# Send the text in pieces of 2,000 characters
chunk_size = 2000
for i in range(0, len(text), chunk_size):
chunk = text[i:i + chunk_size]
await ws.send(json.dumps({
"type": "text.delta",
"delta": chunk
}))
# Signal the end of the text
await ws.send(json.dumps({"type": "text.done"}))
# Receive the audio pieces
async for message in ws:
data = json.loads(message)
if data["type"] == "audio.delta":
audio_bytes = base64.b64decode(data["delta"])
audio_chunks.append(audio_bytes)
elif data["type"] == "audio.done":
print(f"Generation finished (trace: {data['trace_id']})")
break
elif data["type"] == "error":
raise Exception(f"TTS error: {data['message']}")
# Assemble the complete audio
return b"".join(audio_chunks)Implementation in JavaScript (Node.js)
const WebSocket = require("ws");
function streamTTS(apiKey, text, voice = "eve", language = "en") {
return new Promise((resolve, reject) => {
const ws = new WebSocket(
`wss://api.x.ai/v1/tts?language=${language}&voice=${voice}&codec=mp3`,
{ headers: { Authorization: `Bearer ${apiKey}` } }
);
const audioChunks = [];
ws.on("open", () => {
// Send the text
ws.send(JSON.stringify({ type: "text.delta", delta: text }));
ws.send(JSON.stringify({ type: "text.done" }));
});
ws.on("message", (raw) => {
const data = JSON.parse(raw);
if (data.type === "audio.delta") {
audioChunks.push(Buffer.from(data.delta, "base64"));
} else if (data.type === "audio.done") {
ws.close();
resolve(Buffer.concat(audioChunks));
} else if (data.type === "error") {
ws.close();
reject(new Error(data.message));
}
});
ws.on("error", reject);
});
}Handling many deltas
For text generated progressively (an LLM’s output, for example), send each token straight to the TTS WebSocket:
async def llm_to_speech(ws_tts, llm_stream):
async for token in llm_stream:
await ws_tts.send(json.dumps({
"type": "text.delta",
"delta": token
}))
await ws_tts.send(json.dumps({"type": "text.done"}))This “streaming LLM into streaming TTS” architecture offers the lowest possible latency: the user hears the answer while the LLM is still generating.
Key takeaways
- Four client messages:
text.delta(send text),text.done(end of text),text.clear(cancel), andsession.update(replacetable) - Five server messages:
audio.delta(a piece of audio),audio.done(end),audio.clear(cancellation confirmed),session.updated, anderror - Audio is base64-encoded in
audio.deltamessages - The server starts generating before it receives
text.done - Each text delta is limited to 60,000 characters
- Chaining a streaming LLM into streaming TTS gives the lowest latency