Best Practices and Production
On this page
From Proof of Concept to Production
You now have all the building blocks to construct function calling systems with Mistral. What separates your prototype from a production-ready service isn’t missing features, but a series of checks and tuning steps you typically discover during an incident. This final lesson brings together best practices, common pitfalls, and a checklist to move to production with confidence.
Deployment Checklist
This is the only place in this course where a list format is mandatory: before going live, you check boxes. Four domains must be covered, each answering a different question.
Architecture ensures the mechanics hold up. Returning JSON strings in the chain and having a complete dispatch dictionary are the two most frequent causes of crashes on the first real call.
- All functions return JSON strings (never raw Python objects)
- The dispatch dictionary is complete and up to date
- Functions handle error cases and return usable messages
- The system message is clear, precise, and tested
- The conversation loop handles sequential and parallel calls
Security assumes arguments come from a user, never from your application.
- Arguments are validated before execution (format, type, values)
- SQL queries use parameters (no concatenation)
- A whitelist limits available functions
- Destructive actions require confirmation
- Rate limiting is in place per function and per user
- Results are sanitized (no sensitive data returned to the model)
Reliability handles what fails without anyone being at fault.
- Errors are returned to the model in a structured format
- Retry with backoff is implemented for transient errors
- Timeouts are configured for each external function
- History size is limited to avoid token overruns
Monitoring, finally, is what lets you diagnose without guessing.
- Every function call is logged (name, arguments, result, duration)
- Errors trigger alerts (Slack, PagerDuty, etc.)
- Metrics are collected (success rate, latency, API cost)
Writing Descriptions That Make the Model Work
Description quality has a direct impact on accuracy, and it’s often the cheapest lever to pull. Three patterns recur among teams with high correct call rates. The first anchors the description in business context: instead of paraphrasing the function name, list possible return values and explicitly state when to call it. The second slips format examples into the parameter description, which fixes most ID extraction errors. The third replaces a natural language instruction with an enum, the only way to truly forbid a value outside the list.
# Pattern 1: Description with business context
{
"description": "Retrieves payment status. "
"Possible statuses: paid, pending, failed, refunded. "
"Use this function when the user asks "
"about a payment's status or whether it was received."
}
# Pattern 2: Parameters with examples
{
"transaction_id": {
"type": "string",
"description": "Unique ID in format T followed by 4 digits (e.g., T1001, T2345)"
}
}
# Pattern 3: Enum for limited values
{
"status": {
"type": "string",
"enum": ["paid", "pending", "failed", "refunded"],
"description": "Filter by payment status"
}
}Choosing the Right Model
Model choice trades off accuracy against cost, and there’s no universal answer: it depends on the number of exposed functions and the complexity of their schemas. A complex case with many functions and nested parameters justifies Mistral Large 3, which is more reliable at function selection but more expensive. A standard case with a few simple functions in high-throughput production runs very well on Mistral Small 4, which is fast and economical. For prototyping and testing, Mistral Medium 3.5 offers a good compromise.
# Complex cases: many functions, nested parameters
# -> Mistral Large 3 (more reliable, more expensive)
model = "mistral-large-latest"
# Standard cases: few simple functions, high-throughput production
# -> Mistral Small 4 (fast, economical)
model = "mistral-small-latest"
# Development cases: prototyping, testing
# -> Mistral Medium 3.5 (good compromise)
model = "mistral-medium-latest"The method matters as much as the result: in production, start with the most capable model, measure your correct call rate, then step down gradually to find the best cost/performance ratio. The reverse—starting with the cheapest model—deprives you of the baseline needed to know whether degradation comes from the model or your descriptions.
Structuring the System Message
A good system message makes the difference between a mediocre assistant and an excellent one. This one illustrates a proven structure: a role statement, a block of imperative rules, then a reminder of available tools. The two most cost-effective rules here are “If the user doesn’t provide a required ID, ask politely” and “Never guess a transaction ID”—together, they eliminate the most damaging behavior of all, calling a function with a made-up ID.
system_message = {
"role": "system",
"content": """You are a payment management assistant for Acme Corporation.
Rules:
- Always respond in English
- Format amounts in euros with the symbol and 2 decimal places
- If the user doesn't provide a required ID, ask politely
- Never guess a transaction ID
- When a transaction fails, offer to check the details
Available tools:
- retrieve_payment_status: to check a payment's status
- retrieve_payment_date: to check a payment's date
- retrieve_payment_details: to get all details of a transaction
- list_customer_transactions: to view all transactions for a customer"""
}Managing Costs in Production
Every function calling call consumes tokens, and the bill climbs through two underestimated channels: tool definitions are sent with every request, and history grows with each turn. Filtering functions by context solves the first point—no need to send all twelve tools when the question is clearly about payments. Truncating history beyond a threshold solves the second. And when all necessary data is already in context, tool_choice="none" avoids an extra round trip.
# 1. Limit the number of functions per request
# Pass only functions relevant to the context
def get_relevant_tools(user_input: str, all_tools: list) -> list:
"""Filters tools based on context."""
if "payment" in user_input.lower() or "transaction" in user_input.lower():
return [t for t in all_tools if "payment" in t["function"]["name"]]
if "contact" in user_input.lower() or "customer" in user_input.lower():
return [t for t in all_tools if "contact" in t["function"]["name"]]
return all_tools
# 2. Limit history size
MAX_HISTORY = 20 # messages
# 3. Use tool_choice="none" when data is already availableTesting Function Calling
Tests are essential to avoid regressions, and here they target an unusual object: you’re not checking your function’s return value, but the model’s decision. Two test families cover the essentials. The first verifies that a business question triggers the right function with the right extracted arguments. The second, equally important and often forgotten, verifies that a greeting triggers no call—this test catches overly broad descriptions that push the model to call a function in every circumstance.
def test_payment_status():
"""Tests that the model calls the right function."""
messages = [{"role": "user", "content": "Status of payment T1001?"}]
response = client.chat.complete(
model=model, messages=messages, tools=tools
)
message = response.choices[0].message
assert message.tool_calls is not None
assert len(message.tool_calls) == 1
assert message.tool_calls[0].function.name == "retrieve_payment_status"
args = json.loads(message.tool_calls[0].function.arguments)
assert args["transaction_id"] == "T1001"
def test_no_function_for_greeting():
"""Tests that the model doesn't call a function for a greeting."""
messages = [{"role": "user", "content": "Hello!"}]
response = client.chat.complete(
model=model, messages=messages, tools=tools
)
message = response.choices[0].message
assert message.tool_calls is None or len(message.tool_calls) == 0
assert message.content is not NoneMonitoring in Production
Now you need to know what your system does once released into the wild. The class below aggregates three signals that, together, tell almost the whole story: overall success rate, average latency, and breakdown by function. The last one is most telling in practice—a function whose error counter drifts in isolation points to a specific problem, while a degraded overall rate tells you nothing about where to look.
import time
import logging
logger = logging.getLogger("fc_monitor")
class FunctionCallingMonitor:
def __init__(self):
self.metrics = {
"total_calls": 0,
"successful_calls": 0,
"failed_calls": 0,
"total_latency_ms": 0,
"calls_by_function": {}
}
def record(self, function_name: str, success: bool, latency_ms: float):
self.metrics["total_calls"] += 1
self.metrics["total_latency_ms"] += latency_ms
if success:
self.metrics["successful_calls"] += 1
else:
self.metrics["failed_calls"] += 1
if function_name not in self.metrics["calls_by_function"]:
self.metrics["calls_by_function"][function_name] = {
"count": 0, "errors": 0
}
self.metrics["calls_by_function"][function_name]["count"] += 1
if not success:
self.metrics["calls_by_function"][function_name]["errors"] += 1
def get_success_rate(self) -> float:
if self.metrics["total_calls"] == 0:
return 0.0
return self.metrics["successful_calls"] / self.metrics["total_calls"]
def get_avg_latency(self) -> float:
if self.metrics["total_calls"] == 0:
return 0.0
return self.metrics["total_latency_ms"] / self.metrics["total_calls"]
monitor = FunctionCallingMonitor()Next Steps
Now that you’ve mastered function calling with Mistral, several directions open up. The most natural is building autonomous agents, combining function calling with an agent loop that plans and executes complex tasks; the Mistral Agents API and its client.beta.agents.create() go further by offering persistent agents with state. Another axis combines RAG and function calling: the model decides itself when to query a vector database, instead of search being triggered systematically upstream. On the user experience side, streaming lets you display responses in real time while functions execute, changing the perception of latency on long call chains. Finally, multimodal opens function calling to vision for document analysis.
Key Takeaways
- The deployment checklist covers architecture, security, reliability, and monitoring
- Precise descriptions with examples and enums improve model accuracy
- Choose the right model based on complexity and budget (Large for reliability, Small for throughput)
- The system message is a powerful lever to guide behavior
- Test function calls and no-function behavior systematically
- Monitor success rate, latency, and costs in production
Test Your Knowledge
End-to-end function calling: five checks.
1. What are the five steps of function calling?
Answer: Define functions (schemas), provide them to the API, the model requests a call with its arguments, your code executes and returns the result, the model produces the final response.
2. What makes a good function schema?
Answer: Strict types and operational descriptions—clear name, when to use it, format of each parameter: this is the text the model reads to decide and fill in.
3. What do parallel calls enable?
Answer: The model requests multiple functions in a single response when they’re independent—fewer round trips, provided you match each result to its call.
4. What is tool_choice for?
Answer: To set the model’s freedom: auto (it decides), any (it must use a tool), or a forced tool—the last being key to reliable structured extraction.
5. What security rules for exposed functions?
Answer: Validate received arguments, limit each function’s scope, log calls, and keep human validation on committing actions—the model proposes, your code disposes.
The two worked examples in this course (payment tracker, CRM) apply exactly these rules—your own integrations will follow the same mold.