Voice function calling

Run functions from a voice conversation

Voice function calling lets your agent trigger real actions during a conversation: query a database, place an order, send an email, or call an external API. The agent decides when to call a function based on what the user says.

Declare a function

You declare functions in the tools array of session.update:

{
  "type": "session.update",
  "session": {
    "tools": [
      {
        "type": "function",
        "name": "get_balance",
        "description": "Gets the balance of a customer's bank account",
        "parameters": {
          "type": "object",
          "properties": {
            "client_id": {
              "type": "string",
              "description": "Customer ID"
            }
          },
          "required": ["client_id"]
        }
      }
    ]
  }
}

Receive and handle a function call

When the agent decides to call a function, you receive the arguments as a stream, then all at once:

let functionArgs = "";

ws.on("message", (data) => {
  const event = JSON.parse(data);

  switch (event.type) {
    case "response.function_call_arguments.delta":
      functionArgs += event.delta;
      break;

    case "response.function_call_arguments.done":
      const args = JSON.parse(event.arguments);
      const callId = event.call_id;
      const functionName = event.name;

      // Run the function
      handleFunctionCall(functionName, args, callId);
      break;
  }
});

async function handleFunctionCall(name, args, callId) {
  let result;

  switch (name) {
    case "get_balance":
      result = await getBalance(args.client_id);
      break;
    default:
      result = { error: "Unknown function" };
  }

  // Send back the result
  ws.send(JSON.stringify({
    type: "conversation.item.create",
    item: {
      type: "function_call_output",
      call_id: callId,
      output: JSON.stringify(result)
    }
  }));

  // Ask the agent to continue
  ws.send(JSON.stringify({
    type: "response.create"
  }));
}

Parallel function calls

The agent may decide to call several functions at once. You then receive several function_call_arguments.done events. The rule is:

1. Resolve every function before responding
2. Submit every result, with one conversation.item.create per result
3. Send a single response.create at the end

const pendingCalls = new Map();
let expectedCalls = 0;

case "response.function_call_arguments.done":
  const result = await executeFunction(event.name, event.arguments);
  pendingCalls.set(event.call_id, result);

  if (pendingCalls.size === expectedCalls) {
    // Every function is resolved
    for (const [callId, output] of pendingCalls) {
      ws.send(JSON.stringify({
        type: "conversation.item.create",
        item: {
          type: "function_call_output",
          call_id: callId,
          output: JSON.stringify(output)
        }
      }));
    }
    // A single response.create for all the outputs
    ws.send(JSON.stringify({ type: "response.create" }));
    pendingCalls.clear();
  }
  break;

Best practices

• Clear descriptions: the quality of your function descriptions determines when the agent uses them
• Fast execution: a function that takes more than 2 or 3 seconds creates an awkward silence
• Error handling: return a readable error message rather than a technical exception
• Spoken confirmation: for sensitive actions (payments, deletions), have the user confirm before you run them

Key takeaways

• Voice function calling follows a four-step cycle: detection, execution, returning the result, continuation
• You declare functions in session.update with a JSON schema for their parameters
• The result must be sent back through conversation.item.create with type: "function_call_output"
• Always send response.create after submitting the results
• For parallel calls, wait for every result before sending a single response.create