Parallel Function Calls

When the model needs multiple pieces of information at once

So far, every turn in the conversation has resulted in a single function call. Many real-world questions don’t fit that mold. Consider a user who asks, “What is the status and date of payment T1001?” The model needs two pieces of information that live in two distinct functions: retrieve_payment_status and retrieve_payment_date. It could request them one after the other, waiting for your response in between. However, it determines here that neither depends on the other, so it generates multiple tool_calls in a single response. You receive the entire workload at once, and you decide how to execute it.

Reading a response with multiple calls

The practical implication is simple: message.tool_calls is no longer a single-item list. You must iterate over it rather than grabbing tool_calls[0], a common reflex when learning function calling from single-function examples.

response = client.chat.complete(
    model="mistral-large-latest",
    messages=messages,
    tools=tools
)

message = response.choices[0].message

# Multiple tool_calls in the same response
for tc in message.tool_calls:
    print(f"Function: {tc.function.name}")
    print(f"Arguments: {tc.function.arguments}")
    print(f"ID: {tc.id}")
    print("---")

# Possible output:
# Function: retrieve_payment_status
# Arguments: {"transaction_id": "T1001"}
# ID: call_abc123
# ---
# Function: retrieve_payment_date
# Arguments: {"transaction_id": "T1001"}
# ID: call_def456

Note that both calls use the same argument, T1001, but have different IDs: call_abc123 and call_def456. These IDs are the only way the model knows which result corresponds to which request.

Returning each result to its original call

The rule is strict: you must execute every function and return every result in a separate tool message carrying the corresponding tool_call_id. If you forget or swap an ID, the model might attribute the date to the status field, or it may refuse to answer.

message = response.choices[0].message
messages.append(message)

# Execute all requested functions
for tool_call in message.tool_calls:
    function_name = tool_call.function.name
    function_params = json.loads(tool_call.function.arguments)

    # Execute the function
    result = available_functions[function_name](**function_params)

    # Add the result with the correct tool_call_id
    messages.append({
        "role": "tool",
        "name": function_name,
        "content": result,
        "tool_call_id": tool_call.id
    })

# Send all results back to the model
final_response = client.chat.complete(
    model="mistral-large-latest",
    messages=messages,
    tools=tools
)

print(final_response.choices[0].message.content)
# → "Payment T1001 was made on March 15, 2026, and its status is: paid."

The model then stitches the two fragments into a single sentence, as if the information came from one source. This is exactly the effect you want from the user’s perspective.

Moving from apparent parallelism to real parallelism

One nuance often escapes readers on the first pass: in the for loop above, the functions execute one after the other. The model requested its calls in parallel, but your code remains sequential. As long as your functions read an in-memory DataFrame, this doesn’t matter. However, once they query an external API or database, two 300 ms calls become 600 ms of perceived latency, and the benefit of parallelism evaporates. If your functions are asynchronous, asyncio.gather launches them together and waits for the whole batch.

import asyncio
import json

async def execute_tools_parallel(tool_calls, available_functions):
    """Executes tool_calls in parallel and returns the results."""

    async def execute_one(tool_call):
        function_name = tool_call.function.name
        function_params = json.loads(tool_call.function.arguments)
        # If your functions are async:
        result = await available_functions[function_name](**function_params)
        return {
            "role": "tool",
            "name": function_name,
            "content": result,
            "tool_call_id": tool_call.id
        }

    # Execute all functions in parallel
    tasks = [execute_one(tc) for tc in tool_calls]
    return await asyncio.gather(*tasks)

Most existing codebases expose synchronous functions that you won’t want to rewrite as async just for this. A thread pool produces the same effect without touching your business logic.

from concurrent.futures import ThreadPoolExecutor

def execute_tools_threaded(tool_calls, available_functions):
    """Executes tool_calls using a thread pool."""
    def execute_one(tool_call):
        function_name = tool_call.function.name
        function_params = json.loads(tool_call.function.arguments)
        result = available_functions[function_name](**function_params)
        return {
            "role": "tool",
            "name": function_name,
            "content": result,
            "tool_call_id": tool_call.id
        }

    with ThreadPoolExecutor(max_workers=5) as executor:
        results = list(executor.map(execute_one, tool_calls))

    return results

Regaining control with parallel_tool_calls

Parallelism isn’t always desirable. The parallel_tool_calls parameter lets you allow or forbid it. When set to True (the default), the model decides whether to group its calls. When set to False, it can produce only one tool_call per response.

# Allow parallel calls (default)
response = client.chat.complete(
    model="mistral-large-latest",
    messages=messages,
    tools=tools,
    parallel_tool_calls=True  # Model decides
)

# Force sequential calls
response = client.chat.complete(
    model="mistral-large-latest",
    messages=messages,
    tools=tools,
    parallel_tool_calls=False  # Only one tool_call per response
)

Three situations justify setting this to False. The first is when execution order matters: billing before checking stock yields a different result than the reverse. The second involves data dependencies, where the result of one function is required to call the next—for example, finding a contact to get their ID before modifying their record. The third simply relates to the need for finer control over the flow, typically during debugging, when you want to observe each call in isolation before opening the floodgates.

What parallelism really changes

The gain isn’t just about threads; it primarily concerns the number of round trips to the API, which dominate total latency.

# Sequential (2 API calls to the model)
user → assistant(fc.1) → tool(r.1) → assistant(fc.2) → tool(r.2) → assistant(response)

# Parallel (only 1 additional API call)
user → assistant(fc.1, fc.2) → tool(r.1), tool(r.2) → assistant(response)

In the sequential pattern, the model must complete two full reasoning turns before the final answer. In the parallel pattern, a single request retrieves all tool_calls, and only one additional request is needed for the response. For a question involving two or three functions, this commonly saves the user several seconds.

Key takeaways

• The model can generate multiple tool_calls in a single response
• Each result must be returned with the correct tool_call_id
• For network functions, use asyncio or ThreadPoolExecutor for true parallel execution
• The parallel_tool_calls parameter controls this behavior (True by default)
• The parallel pattern reduces API requests and improves latency