Tools and function calling in voice

An agent that acts, not just talks

A voice assistant that can only chat soon hits a wall: it answers “Where’s my order?” politely, without knowing a thing. The real value of the Voice Agent lies in its ability to take action in the middle of a conversation (query a database, run a web search, call a business API, reach an MCP server) and read out the result as if nothing happened.

The five kinds of tools

Web search, in its simplest form, takes no parameters:

{ "type": "web_search" }

The agent then searches the web live, which covers factual questions, the news, and any data that changes too often to live in a prompt.

X search works the same way, with the option to limit it to certain accounts:

{
  "type": "x_search",
  "allowed_x_handles": ["corsen_ai"]
}

The allowed_x_handles parameter is optional, but it changes what the tool is. Without it, the agent searches the whole platform; with it, the agent becomes a spokesperson that only quotes your official posts.

File search queries collections of vectorized documents, such as your manuals, procedures, and product documentation:

{
  "type": "file_search",
  "vector_store_ids": ["vs_abc123"],
  "max_num_results": 10
}

Custom functions open the door to your own systems. This is the kind of tool you’ll use most, and it’s worth declaring carefully:

{
  "type": "function",
  "name": "check_order",
  "description": "Checks the status of a customer order",
  "parameters": {
    "type": "object",
    "properties": {
      "order_number": {
        "type": "string",
        "description": "The order number, in the format CMD-XXXX"
      }
    },
    "required": ["order_number"]
  }
}

You can connect a CRM, an ERP, a calendar, or an internal database the same way. Finally, MCP servers give access to standardized tools through the Model Context Protocol, so you don’t have to rewrite an integration for every service:

{
  "type": "mcp",
  "server_url": "https://mcp.example.com/sse",
  "server_label": "My MCP server"
}

The full function call cycle

When the agent decides to call a function, it doesn’t run it: it asks you to, and your app does the work. The arguments first stream in through response.function_call_arguments.delta. Then the response.function_call_arguments.done event marks the end of the request and gives you the call_id along with the complete arguments. It’s up to you to run the function, then return the result as a conversation.item.create of type function_call_output:

{
  "type": "conversation.item.create",
  "item": {
    "type": "function_call_output",
    "call_id": "call_abc123",
    "output": "{\"status\": \"shipped\", \"delivery_date\": \"2026-04-05\"}"
  }
}

One last step is essential, and it’s the one people forget: send response.create so the agent works the result into its spoken reply. Without it, the result does reach the conversation but nothing happens. The user hears silence, and you go looking for the bug in the wrong place.

When several functions fire at once

The model may well decide to call two or three functions at once, for example to check an order and look up the delivery schedule. You then receive several function_call_arguments.done events, each with its own call_id. Run them all, in parallel if you can so the latencies don’t add up, submit each result in its own conversation.item.create, and send a single response.create at the very end. One response.create per function would trigger as many competing voice responses, and the agent would start talking over itself.

// Receive and run all the functions
const results = await Promise.all(
  pendingCalls.map(call => executeFunction(call))
);

// Submit all the results
for (const result of results) {
  ws.send(JSON.stringify({
    type: "conversation.item.create",
    item: {
      type: "function_call_output",
      call_id: result.callId,
      output: JSON.stringify(result.data)
    }
  }));
}

// Trigger the response
ws.send(JSON.stringify({ type: "response.create" }));

Constraints specific to voice

What goes unnoticed in a text interface becomes glaring in speech, starting with time. Nobody waits more than three to five seconds in the middle of a conversation. Beyond that, users think the line has dropped and start talking. If your function queries a slow system, have the agent say a holding phrase during the call instead of leaving dead air.

The size of the results matters just as much. The agent has to summarize out loud whatever you give it, and a list of forty orders produces either an endless recital or a rough summary. Return structured data, but keep it to what matters. Take care with each function’s description too, since it’s the only thing the model relies on to choose: a vague description like “handles orders” makes it hesitate or pick the wrong tool. And plan for failure. When a function crashes, return an explicit error message in output rather than nothing, or the agent will improvise an answer without knowing the call failed.

Key takeaways

• The Voice Agent supports five types of tools: web search, X search, file search, functions, and MCP
• A function call goes through four steps: receive the arguments, run the function, return the result, and trigger the response
• Parallel calls are possible: run everything, submit everything, then send a single response.create
• In voice, function response time is critical: aim for under 3 seconds