Controlling Prosody and Speech Rate

Updated November 2, 2026 — Audio models. Several models in this course are deprecated: tts-1, tts-1-hd, and gpt-4o-mini-tts will stop working on January 6, 2027 (successor: gpt-realtime-2.1-mini, which the OpenAI TTS guide states is not supported by /v1/audio/speech: voice synthesis migration moves to the Realtime API); gpt-realtime, gpt-4o-realtime, and legacy audio models on January 20, 2027 (gpt-realtime-2.1, gpt-realtime-2.1-mini, gpt-audio-1.5); whisper-1, gpt-4o-transcribe, and /v1/audio/translations on February 26, 2027 (gpt-transcribe, gpt-live-transcribe, v1/realtime/translations). For full-duplex voice, GPT-Live 1 has been available in the API since September 10, 2026. Lesson completed November 3, 2026: it previously omitted voice instructions for gpt-4o-mini-tts (instructions), the most direct lever for intonation, tone, and speech rate, which the documentation describes and which tts-1 does not accept. Source: Text to speech.

Prosody—intonation, rhythm, stress, and pauses—is what separates a robotic synthetic voice from a natural one. OpenAI’s TTS API does not support SSML, but it offers two direct levers: the instructions parameter for gpt-4o-mini-tts and the speed parameter, supplemented by text formatting techniques.

Voice Instructions for `gpt-4o-mini-tts`

With gpt-4o-mini-tts, you describe how to speak in natural language using the instructions parameter. The documentation lists what these instructions control: accent, emotional range, intonation, impersonations, speech rate, tone, and whispering.

from openai import OpenAI

client = OpenAI()

with client.audio.speech.with_streaming_response.create(
    model="gpt-4o-mini-tts",
    voice="marin",
    input="Your appointment is confirmed for tomorrow at two PM.",
    instructions=(
        "Calm and reassuring voice of a medical receptionist. Measured pace, "
        "short pause before the time, falling intonation at the end of the sentence."
    ),
) as response:
    response.stream_to_file("appointment.mp3")

Keep the text to speak in input and the delivery style in instructions. Do not mix them, or you risk hearing the instruction itself spoken aloud. These instructions do not work with tts-1 or tts-1-hd; with those models, only speed and text formatting, discussed below, affect the output. Since all three models sunset on January 6, 2027, migration to the Realtime API (Lesson 10) follows this same principle: tone instructions move into session or response instructions.

Controlling Speech Rate with the Speed Parameter

The TTS API provides a speed parameter to adjust speaking rate:

from openai import OpenAI

client = OpenAI()

# Normal speed (1.0)
response_normal = client.audio.speech.create(
    model="tts-1",
    voice="nova",
    input="Your appointment is confirmed for tomorrow at two PM.",
    speed=1.0
)

# Slower (0.75) — for complex instructions
response_slow = client.audio.speech.create(
    model="tts-1",
    voice="nova",
    input="Your appointment is confirmed for tomorrow at two PM.",
    speed=0.75
)

# Faster (1.25) — for short notifications
response_fast = client.audio.speech.create(
    model="tts-1",
    voice="nova",
    input="Your appointment is confirmed for tomorrow at two PM.",
    speed=1.25
)

Speed	Range	Use Case
Slow	0.25 — 0.75	Complex instructions, accessibility, elderly users
Normal	0.9 — 1.1	Conversations, training, narration
Fast	1.25 — 2.0	Notifications, confirmations, summaries

Text Formatting Techniques

The primary lever for controlling prosody is formatting the input text. The TTS model interprets punctuation and structure to adapt its rate and intonation.

Pauses

# Short pause: comma
text_comma = "First step, check your configuration."

# Medium pause: semicolon or em dash
text_semi = "First step — check your configuration."

# Long pause: period or ellipsis
text_dots = "First step. Check your configuration."
text_ellipsis = "First step... check your configuration."

# Very long pause: separate paragraph
text_para = "First step.\n\nCheck your configuration."

Emphasis and Intonation

# Question (rising intonation)
text_question = "Would you like to confirm this reservation?"

# Exclamation (natural emphasis)
text_exclaim = "Congratulations! Your registration is confirmed!"

# Emphasis via capitalization
text_caps = "It is VERY important not to skip this step."

# List with rhythm (the model naturally marks each item)
text_list = "You will need: one, your username. Two, your password. Three, your confirmation code."

Adapting Text for Voice Context

Text written for reading and text written for speaking are very different. For a voice agent, systematically reformat content:

def format_for_speech(text: str) -> str:
    """Adapts written text for natural voice playback."""
    import re

    # Spell out common acronyms
    abbreviations = {
        "API": "A P I",
        "URL": "U R L",
        "PDF": "P D F",
        "SMS": "S M S",
    }
    for abbr, spelled in abbreviations.items():
        text = text.replace(abbr, spelled)

    # Write small numbers as words
    number_words = {
        "1": "one", "2": "two", "3": "three", "4": "four",
        "5": "five", "6": "six", "7": "seven", "8": "eight",
        "9": "nine", "10": "ten"
    }
    for digit, word in number_words.items():
        text = re.sub(rf'\b{digit}\b', word, text)

    # Write times in spoken format
    text = re.sub(r'(\d{1,2})h(\d{2})', r'\1 hours \2', text)
    text = re.sub(r'(\d{1,2})h\b', r'\1 hours', text)

    return text

original = "Your appt is at 14h30. Send 1 SMS to confirm."
formatted = format_for_speech(original)
# → "Your appt is at 14 hours 30. Send one S M S to confirm."

Adaptive Speed Based on Content

An intelligent voice agent adapts its speech rate to the content type:

def adaptive_speed(content_type: str, user_preference: float = 1.0) -> float:
    """Calculates optimal speed based on content type."""
    base_speeds = {
        "greeting": 1.0,
        "instructions": 0.85,
        "confirmation": 1.1,
        "error_message": 0.9,
        "phone_number": 0.7,
        "address": 0.75,
        "summary": 1.15,
        "farewell": 1.0,
    }

    base = base_speeds.get(content_type, 1.0)
    # Adjust with user preference (0.5 to 1.5)
    adjusted = base * user_preference
    return max(0.25, min(4.0, adjusted))  # Clamp to API limits

# Usage
speed = adaptive_speed("phone_number", user_preference=0.9)
# → 0.63 (slow for dictating a number)

Audio Post-Processing

For even finer control, process the audio after generation:

from pydub import AudioSegment

def post_process_speech(audio_bytes: bytes, adjustments: dict) -> bytes:
    """Audio post-processing for fine adjustments."""
    import io
    audio = AudioSegment.from_mp3(io.BytesIO(audio_bytes))

    # Adjust volume
    if "volume_db" in adjustments:
        audio = audio + adjustments["volume_db"]

    # Add fade in/out
    if adjustments.get("fade_in_ms"):
        audio = audio.fade_in(adjustments["fade_in_ms"])
    if adjustments.get("fade_out_ms"):
        audio = audio.fade_out(adjustments["fade_out_ms"])

    # Add silence at start (pre-roll)
    if adjustments.get("pre_silence_ms"):
        silence = AudioSegment.silent(duration=adjustments["pre_silence_ms"])
        audio = silence + audio

    buffer = io.BytesIO()
    audio.export(buffer, format="mp3")
    return buffer.getvalue()

Key Takeaways

With gpt-4o-mini-tts, instructions describes how to speak (tone, intonation, rate, accent); tts-1 and tts-1-hd do not accept this parameter
The speed parameter (0.25 to 4.0) controls overall speech rate
Punctuation is the primary lever for pauses: comma (short), period (long), ellipsis (dramatic)
Adapt text for voice context: spell out acronyms, write numbers as words, rephrase times
Vary speed by content: slow for numbers and addresses, fast for confirmations
Audio post-processing (volume, fades, silence) refines the final output

Controlling Prosody and Speech Rate