Pipeline ta' streaming mill-LLM għat-TTS
F'din il-paġna
Aġġornat fit-3 ta’ Novembru, 2026, ibbażat fuq il-paġna Text to Speech ta’ xAI: kull
text.deltajista’ jżomm sa 60,000 karattru. Sors: Text to Speech.
Qabbad LLM mas-sinteżi tad-dawl f’ħin reali
L-aktar użu qawwi tal-WebSocket tat-TTS huwa li tgħaqqdu ma’ LLM li jixxandar (streaming). L-LLM jiġġenera t-test token b’token, kull token imur għat-TTS permezz tal-WebSocket, u l-awdjo jindaqq hekk kif jasal. L-utent jisma’ t-tweġiba waqt li l-LLM għadu qed jaħseb.
Arkitettura tal-pipeline
Id-data tgħaddi minn tliet stadji:
User -> LLM (streaming) -> TTS WebSocket -> Audio (playback)
question text tokens audio.delta speaker
Kull stadju jixxandar, li jimminimizza l-latenza perċepita. L-utent ma jistenniex sakemm l-LLM jispiċċa jiġġenera t-tweġiba kollha tiegħu qabel ma jibda jisma’.
Implimentazzjoni f’Python
Hawnhekk hemm pipeline sħiħa li tgħaqqad l-API ta’ Grok Chat Completions mal-WebSocket tat-TTS:
import asyncio
import websockets
import json
import base64
import aiohttp
async def llm_to_speech(api_key, prompt, voice="rex", language="en"):
# Pass 1: Konnessjoni TTS WebSocket
tts_uri = f"wss://api.x.ai/v1/tts?language={language}&voice={voice}"
tts_headers = {"Authorization": f"Bearer {api_key}"}
audio_chunks = []
async with websockets.connect(tts_uri, additional_headers=tts_headers) as tts_ws:
# Pass 2: Sejħa streaming tal-LLM
async with aiohttp.ClientSession() as session:
async with session.post(
"https://api.x.ai/v1/chat/completions",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json"
},
json={
"model": "grok-4.5",
"messages": [{"role": "user", "content": prompt}],
"stream": True
}
) as llm_response:
# Pass 3: Kull token tal-LLM -> delta tat-TTS
buffer = ""
async for line in llm_response.content:
line = line.decode().strip()
if not line.startswith("data: "):
continue
if line == "data: [DONE]":
break
chunk = json.loads(line[6:])
token = chunk["choices"][0].get("delta", {}).get("content", "")
if token:
buffer += token
# Ibgħat sentenzi sħaħ
if any(c in token for c in ".!?\n"):
await tts_ws.send(json.dumps({
"type": "text.delta",
"delta": buffer
}))
buffer = ""
# Ibgħat kwalunkwe ħaġa li fadal fil-buffer
if buffer:
await tts_ws.send(json.dumps({
"type": "text.delta",
"delta": buffer
}))
await tts_ws.send(json.dumps({"type": "text.done"}))
# Pass 4: Iġbor l-awdjo
async for msg in tts_ws:
data = json.loads(msg)
if data["type"] == "audio.delta":
audio_chunks.append(base64.b64decode(data["delta"]))
elif data["type"] == "audio.done":
break
return b"".join(audio_chunks)
Strateġija ta’ buffering
Mhuwiex ideali li tibgħat kull token lit-TTS individwalment: is-mutur tas-sinteżi jipproduċi riżultati aħjar meta jkollu kuntest. Hemm tliet strateġiji ta’ buffering.
Skont is-sentenza
Stenna delimiter tas-sentenza (., !, ?, new line) qabel ma tibgħat:
if any(c in token for c in ".!?\n"):
await tts_ws.send(json.dumps({
"type": "text.delta", "delta": buffer
}))
buffer = ""
Dan huwa l-aħjar bilanċ bejn il-kwalità u l-latenza.
Skont id-daqs
Ibgħat meta l-buffer jilħaq ċertu daqs (200 karattru, pereżempju):
if len(buffer) >= 200:
await tts_ws.send(json.dumps({
"type": "text.delta", "delta": buffer
}))
buffer = ""
Huwa aktar sempliċi iżda jista’ jaqta’ kelma f’nofsha.
Skont il-ħin (timeout)
Ibgħat wara dewmien mingħajr l-ebda token ġdid (utli waqt li l-LLM ikun qed jaħseb):
last_token_time = time.time()
# Jekk jgħaddu aktar minn 500 ms mingħajr token, ibgħat il-buffer
if time.time() - last_token_time > 0.5 and buffer:
await tts_ws.send(...)
buffer = ""
Riproduzzjoni tal-awdjo f’ħin reali
Biex tdaqq l-awdjo hekk kif jasal (mhux biss biex tiġboru), uża queue asinkrona:
import asyncio
audio_queue = asyncio.Queue()
async def receive_audio(tts_ws):
"""Jirċievi biċċiet tal-awdjo u jqiegħdhom fil-queue."""
async for msg in tts_ws:
data = json.loads(msg)
if data["type"] == "audio.delta":
audio_bytes = base64.b64decode(data["delta"])
await audio_queue.put(audio_bytes)
elif data["type"] == "audio.done":
await audio_queue.put(None) # Sentinel tat-tmiem
break
async def play_audio():
"""Idaqq biċċiet tal-awdjo mill-queue."""
while True:
chunk = await audio_queue.get()
if chunk is None:
break
# Daqq il-biċċa (pyaudio, sounddevice, eċċ.)
play_chunk(chunk)
Billi tħaddem receive_audio u play_audio parallelament, ir-riproduzzjoni tibda bl-ewwel biċċa li tiġi rċevuta.
Ġestjoni tal-iżbalji fil-pipeline
Il-pipeline tista’ tfalli f’tliet postijiet: l-LLM, it-TTS, jew in-netwerk. Iġġesti kull wieħed minnhom:
async def robust_pipeline(api_key, prompt):
try:
audio = await llm_to_speech(api_key, prompt)
return audio
except aiohttp.ClientError:
print("Żball fl-LLM: iċċekkja l-konnessjoni tiegħek")
except websockets.ConnectionClosed:
print("Konnessjoni tat-TTS mitlufa: meħtieġa konnessjoni mill-ġdid")
except json.JSONDecodeError:
print("Tweġiba invalida: format mhux mistenni")
except Exception as e:
print(f"Żball mhux mistenni: {e}")
return None
Punti Ewlenin
- LLM li jixxandar li jgħaqqad mat-TTS WebSocket jagħti l-inqas latenza
- Agħmel buffer skont is-sentenza għall-aħjar bilanċ bejn il-kwalità u l-latenza
- Uża queue asinkrona għar-riproduzzjoni tal-awdjo f’ħin reali
- Iġġesti l-iżbalji f’kull stadju tal-pipeline
- It-test totali fuq WebSocket huwa mingħajr limitu (deltas sa 60,000 karattru)
- Il-limitu ta’ 50 sessjoni konkorrenti japplika għat-tim kollu tiegħek