Aqbeż għall-kontenut ewlieni
Malti— Agħżel lingwa
Menu

Strimming tal-awdjo f'ħin reali

Lezzjoni 12 / 18 Aġġornat 3 ta’ Novembru 2026 Zviluppatur
F'din il-paġna
  1. Strimming tal-awdjo f’ħin reali
  2. Strimming mal-API TTS
  3. Kejl tal-latenza
  4. Strimming asinkronu
  5. Buffer ta’ pre-mili
  6. Integrazzjoni ma’ pipeline LLM + TTS
  7. Punti Ewlenin

Aġġornament 2 ta’ Novembru, 2026 — Mudelli tal-awdjo. Diversi mudelli f’dan il-kors huma obsoleti: tts-1, tts-1-hd, u gpt-4o-mini-tts se jieqfu jaħdmu fis-6 ta’ Jannar, 2027 (suċċessur: gpt-realtime-2.1-mini, li l-gwida TTS ta’ OpenAI tiddikjara li mhux appoġġjata minn /v1/audio/speech: il-migrazzjoni tas-sinteżi tal-vuċi tmur lejn l-API f’ħin reali); gpt-realtime, gpt-4o-realtime, u mudelli tal-awdjo antiki fl-20 ta’ Jannar, 2027 (gpt-realtime-2.1, gpt-realtime-2.1-mini, gpt-audio-1.5); whisper-1, gpt-4o-transcribe, u /v1/audio/translations fis-26 ta’ Frar, 2027 (gpt-transcribe, gpt-live-transcribe, v1/realtime/translations). Għal vuċi full-duplex, GPT-Live 1 kien disponibbli fl-API mill-10 ta’ Settembru, 2026. Aġġornat fit-3 ta’ Novembru, 2026: Il-pipeline LLM → TTS qabel kien jipproċessa kwalunkwe avveniment li jġorr kamp delta, inklużi argumenti ta’ sejħiet ta’ funzjonijiet jew sommarji tar-raġunament; issa jiffiltra strettament għal response.output_text.delta, it-test attwali tar-rispons.

Strimming tal-awdjo f’ħin reali

F’aġent tal-vuċi, li tistenna sakemm il-fajl tal-awdjo kollu jiġi ġġenerat qabel ma tibda r-riproduzzjoni tintroduċi latenza inaċċettabbli. L-istrimming TTS jippermettilek tibda r-riproduzzjoni hekk kif jaslu l-ewwel bytes, u tonqos il-ħin għall-ewwel rispons għal ftit mijiet ta’ millisekondi.

Strimming mal-API TTS

L-API TTS tappoġġja l-istrimming nativament. Minflok ma tistenna l-fajl sħiħ, tirċievi l-awdjo f’biċċiet:

from openai import OpenAI
import pyaudio

client = OpenAI()

def stream_tts_to_speaker(text: str, voice: str = "alloy"):
    """Iġġenera u doqq l-awdjo f'modalità ta' strimming."""
    # Inizzjalizza d-dowqq tal-awdjo
    pa = pyaudio.PyAudio()
    player = pa.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=24000,
        output=True
    )

    try:
        # Oħloq ir-rispons tal-istrimming
        with client.audio.speech.with_streaming_response.create(
            model="tts-1",
            voice=voice,
            input=text,
            response_format="pcm"  # PCM mhux ipproċessat għal riproduzzjoni diretta
        ) as response:
            # Aqra u doqq biċċa biċċa
            for chunk in response.iter_bytes(chunk_size=4096):
                player.write(chunk)

    finally:
        player.stop_stream()
        player.close()
        pa.terminate()

stream_tts_to_speaker("Hello, here is a real-time streaming response.")

Il-format pcm huwa ideali għall-istrimming għax ma jeħtieġ l-ebda dekodifika: il-bytes raw huma lesti biex jindaqqu immedjatament.

Kejl tal-latenza

Il-Ħin għall-Ewwel Byte (TTFB) huwa l-metrika ewlenija għall-prestazzjoni tal-istrimming:

import time

def measure_tts_latency(text: str, voice: str = "alloy") -> dict:
    """Kejjel il-latenzi tal-istrimming TTS."""
    start_time = time.time()
    first_byte_time = None
    total_bytes = 0

    with client.audio.speech.with_streaming_response.create(
        model="tts-1",
        voice=voice,
        input=text,
        response_format="pcm"
    ) as response:
        for chunk in response.iter_bytes(chunk_size=4096):
            if first_byte_time is None:
                first_byte_time = time.time()
            total_bytes += len(chunk)

    end_time = time.time()

    return {
        "ttfb_ms": round((first_byte_time - start_time) * 1000),
        "total_ms": round((end_time - start_time) * 1000),
        "total_bytes": total_bytes,
        "audio_duration_s": round(total_bytes / (24000 * 2), 2)
    }

metrics = measure_tts_latency("Hello, how are you?")
print(f"TTFB: {metrics['ttfb_ms']}ms, Total: {metrics['total_ms']}ms")

Strimming asinkronu

Għal applikazzjonijiet fuq is-server, uża l-klijent asinkronu:

from openai import AsyncOpenAI
import asyncio

async_client = AsyncOpenAI()

async def stream_tts_async(text: str, voice: str = "alloy"):
    """Strimming TTS asinkronu għas-servers."""
    chunks = []

    async with async_client.audio.speech.with_streaming_response.create(
        model="tts-1",
        voice=voice,
        input=text,
        response_format="pcm"
    ) as response:
        async for chunk in response.iter_bytes(chunk_size=4096):
            chunks.append(chunk)
            # Hawnhekk, tibgħat il-biċċa lill-klijent permezz ta' WebSocket
            yield chunk

# Użu f'server FastAPI
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

@app.get("/tts/stream")
async def tts_stream(text: str, voice: str = "alloy"):
    return StreamingResponse(
        stream_tts_async(text, voice),
        media_type="audio/pcm"
    )

Buffer ta’ pre-mili

Biex tiżgura riproduzzjoni fluwida mingħajr vojt, akkumula buffer żgħir qabel ma tibda:

async def buffered_tts_playback(
    text: str,
    voice: str = "alloy",
    buffer_size: int = 3  # Numru ta' biċċiet biex jitgħabbew minn qabel
):
    """Riproduzzjoni TTS b'buffer ta' pre-mili."""
    pa = pyaudio.PyAudio()
    player = pa.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=24000,
        output=True
    )

    buffer = []
    playback_started = False

    try:
        async with async_client.audio.speech.with_streaming_response.create(
            model="tts-1",
            voice=voice,
            input=text,
            response_format="pcm"
        ) as response:
            async for chunk in response.iter_bytes(chunk_size=4096):
                buffer.append(chunk)

                if not playback_started and len(buffer) >= buffer_size:
                    playback_started = True
                    # Neħħi l-buffer inizjali
                    for buffered_chunk in buffer:
                        player.write(buffered_chunk)
                    buffer.clear()
                elif playback_started:
                    player.write(chunk)

        # Doqq il-biċċiet li fadal
        for remaining in buffer:
            player.write(remaining)

    finally:
        player.stop_stream()
        player.close()
        pa.terminate()

Integrazzjoni ma’ pipeline LLM + TTS

F’aġent tal-vuċi mhux f’ħin reali (non-Realtime), tista’ tgħaqqad l-istrimming tal-LLM mal-istrimming TTS:

async def llm_to_tts_pipeline(user_message: str):
    """Pipeline: Strimming LLM → Strimming TTS f'ħin reali."""
    # Fażi 1: Akkumula r-rispons tal-LLM skont is-sentenzi
    sentence_buffer = ""

    stream = client.responses.create(
        model="gpt-5.6-luna",
        input=[{"role": "user", "content": user_message}],
        stream=True
    )

    for event in stream:
        # Biss it-test tar-rispons għandu jinkellem
        if event.type == "response.output_text.delta":
            sentence_buffer += event.delta

            # Skopri t-tmiem tas-sentenza
            if sentence_buffer.rstrip().endswith((".", "!", "?")):
                sentence = sentence_buffer.strip()
                sentence_buffer = ""

                # Fażi 2: Ixandar din is-sentenza lejn it-TTS
                stream_tts_to_speaker(sentence)

    # Ipproċessa l-buffer li fadal
    if sentence_buffer.strip():
        stream_tts_to_speaker(sentence_buffer.strip())

Punti Ewlenin

  • L-istrimming TTS inaqqas it-TTFB għal ftit mijiet ta’ millisekondi billi jibda r-riproduzzjoni qabel ma tintemm il-ġenerazzjoni
  • Il-format pcm huwa ottimali għall-istrimming: l-ebda dekodifika meħtieġa, riproduzzjoni diretta
  • Buffer ta’ pre-mili ta’ 2–3 biċċiet jipprevjeni vojt fil-bidu tar-riproduzzjoni
  • Il-klijent asinkronu huwa essenzjali għas-servers li jittrattaw konnessjonijiet simultanji multipli
  • Il-kombinazzjoni ta’ LLM li jixxandar u TTS li jixxandar jippermetti aġent tal-vuċi responsiv mingħajr l-API f’ħin reali (Realtime API)