Azure in Production — Mistral Deployment: Cloud and Self-Hosting

Prepare a Robust Azure Deployment

A working prototype on your laptop and a production system don’t differ in their calling code, which remains identical. They differ in everything surrounding that code: where secrets come from, how you observe what’s happening, what happens when traffic doubles, and which network path data travels. This lesson covers these four dimensions for a Mistral deployment on Azure.

Production Architecture

A typical deployment brings together five building blocks. The Azure AI endpoint hosts the Mistral model as MaaS; Azure Key Vault stores API keys; Azure Monitor and Application Insights provide observability; Azure API Management acts as a gateway with throttling and authentication in front of your application; finally, an Azure Virtual Network provides network isolation, which is optional but often required for sensitive data.

Your first move should be to pull secrets out of your code and environment variables, having the application read them at startup from Key Vault instead.

from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
from mistralai.azure.client import MistralAzure

# Retrieve the API key from Key Vault
credential = DefaultAzureCredential()
vault_url = "https://mon-keyvault.vault.azure.net"
secret_client = SecretClient(vault_url=vault_url, credential=credential)

endpoint = secret_client.get_secret("mistral-endpoint").value
api_key = secret_client.get_secret("mistral-api-key").value

client = MistralAzure(server_url=endpoint, api_key=api_key)

The benefit goes beyond simple confidentiality. When you rotate a key, no application redeployment is needed because the value is read from the vault rather than hardcoded in configuration.

Monitoring with Application Insights

Without instrumentation, the only thing you’ll know about an incident is that a user finds the application “slow.” Tracing every call with its duration, token count, and model gives you what you need to respond within minutes.

import time
import logging
from opencensus.ext.azure.log_exporter import AzureLogHandler

logger = logging.getLogger(__name__)
logger.addHandler(AzureLogHandler(
    connection_string="InstrumentationKey=your-key"
))

def monitored_call(client, messages, model="azureai"):
    """Mistral call with Azure Monitor metrics."""
    start = time.time()
    try:
        response = client.chat.complete(model=model, messages=messages)
        duration = time.time() - start
        logger.info(
            "MistralCall",
            extra={
                "custom_dimensions": {
                    "duration_ms": int(duration * 1000),
                    "prompt_tokens": response.usage.prompt_tokens,
                    "completion_tokens": response.usage.completion_tokens,
                    "model": model
                }
            }
        )
        return response
    except Exception as e:
        duration = time.time() - start
        logger.error(
            f"MistralError: {e}",
            extra={
                "custom_dimensions": {
                    "duration_ms": int(duration * 1000),
                    "error_type": type(e).__name__
                }
            }
        )
        raise

These metrics are only valuable if someone watches them. Configure the corresponding alerts in Azure Monitor:

P95 latency > 5s — performance degradation
5xx error rate > 1% — server-side issues
429 errors > 10/min — rate limit reached
Token consumption > 80% of quota — approaching limit

Scalability

When volume grows, the first mistake is calling the model synchronously from the incoming HTTP request: any slowdown propagates straight to the user. Decoupling with Azure Service Bus lets you absorb spikes—requests pile up in a queue and are consumed at the pace your quota allows.

from azure.servicebus import ServiceBusClient, ServiceBusMessage
import json

# Producer: send requests to the queue
def enqueue_request(prompt: str, request_id: str):
    with ServiceBusClient.from_connection_string(conn_str) as sb_client:
        sender = sb_client.get_queue_sender(queue_name="mistral-requests")
        message = ServiceBusMessage(
            json.dumps({"prompt": prompt, "request_id": request_id})
        )
        sender.send_messages(message)

# Consumer: process requests from the queue
def process_queue():
    with ServiceBusClient.from_connection_string(conn_str) as sb_client:
        receiver = sb_client.get_queue_receiver(queue_name="mistral-requests")
        for msg in receiver:
            data = json.loads(str(msg))
            response = client.chat.complete(
                model="azureai",
                messages=[{"role": "user", "content": data["prompt"]}]
            )
            # Store the result...
            receiver.complete_message(msg)

If the bottleneck comes from throughput itself rather than instantaneous load, deploy multiple endpoints for the same model and distribute calls across them.

import random

endpoints = [
    {"url": os.environ["AZUREAI_ENDPOINT_1"], "key": os.environ["AZUREAI_KEY_1"]},
    {"url": os.environ["AZUREAI_ENDPOINT_2"], "key": os.environ["AZUREAI_KEY_2"]},
]

def get_client():
    """Simple round-robin across endpoints."""
    ep = random.choice(endpoints)
    return MistralAzure(server_url=ep["url"], api_key=ep["key"])

Network Security

For sensitive data, Azure Private Link ensures that traffic between your application and the Mistral endpoint never crosses the public internet. Configuration happens in four steps:

Create a Private Endpoint in your VNet
Link it to your Azure AI resource
Disable public access on the endpoint
Configure private DNS for resolution

Step three is the one people forget, and without it the public path remains open in parallel: Private Link exists, but nothing prevents bypassing it.

In the same spirit, a Managed Identity removes the API key from the equation: the application obtains its tokens from Microsoft Entra ID, so there’s no secret to store or rotate. Two conditions apply, however. The MistralAzure client documented by Mistral authenticates by key; the keyless authentication Microsoft describes goes through a model deployed in a Microsoft Foundry resource, queried on its OpenAI-compatible route /openai/v1/, with tokens issued for the scope https://ai.azure.com/.default. And the identity must have been granted the inference role on the resource beforehand. A token provider, rather than a token read once at startup, renews access before expiration:

from azure.identity import ManagedIdentityCredential, get_bearer_token_provider
from openai import OpenAI

token_provider = get_bearer_token_provider(
    ManagedIdentityCredential(),
    "https://ai.azure.com/.default",
)

client = OpenAI(
    base_url="https://<resource>.openai.azure.com/openai/v1/",
    api_key=token_provider,
)

response = client.chat.completions.create(
    model="my-mistral-deployment",  # name of your deployment
    messages=[{"role": "user", "content": "Summarize the data retention policy."}],
)
print(response.choices[0].message.content)

Source: Endpoints for Microsoft Foundry Models — keyless authentication.

Production Readiness Checklist

Before declaring your deployment ready, walk through the chain in the order it was built. Environment variables and secrets must have migrated to Key Vault, and your calling code must be protected by exponential retry covering both 429 and 5xx errors. Application Insights monitoring must report latency, errors, and tokens, with corresponding alerts actually configured on critical metrics—not just planned. Finally, tackle what people tend to postpone: load tests actually run, private network access if data is sensitive, and API key rotation scheduled with a date, not left to general discretion.

Key Takeaways

Use Azure Key Vault to store API keys, not plaintext environment variables
Monitor every call with Application Insights (latency, tokens, errors)
Decouple with Service Bus for high-volume workloads
Use Private Link and Managed Identity for secure environments
Run load tests and configure alerts before going to production