API Anti-Patterns

Mistakes That Cost Money in Production

After covering best practices, it’s time to look at the flip side. The anti-patterns below are the most common pitfalls when using the Chat Completions API, and they share one unpleasant trait: they don’t show up during development. Out of ten test requests, everything works fine. It’s in production that they blow up, causing unexpected costs, incoherent responses, high latency, or worse—leaks of sensitive data.

Treating a Truncated Response as Complete

The most common mistake is ignoring finish_reason. You ask for a contract, you get some text back, you save it to the database: except max_tokens=100 cut off the generation in the middle of a clause, and no one notices until the client signs. The fix takes three lines of code and should become a reflex.

# BAD — no verification
response = client.chat.complete(
    model="mistral-large-latest",
    messages=[{"role": "user", "content": "Draft a complete contract."}],
    max_tokens=100  # Too short for a contract!
)
contract = response.choices[0].message.content  # Potentially truncated!

# GOOD — systematic verification
response = client.chat.complete(
    model="mistral-large-latest",
    messages=[{"role": "user", "content": "Draft a complete contract."}],
    max_tokens=4000
)

if response.choices[0].finish_reason == "length":
    print("WARNING: Response truncated, increase max_tokens")
else:
    contract = response.choices[0].message.content

Letting Conversation History Grow Without Limits

The second trap triggers slowly. A chat loop that sends the entire conversation history on every turn works perfectly for the first ten exchanges, costs double by the twentieth, and eventually exceeds the context window by the hundredth—with the bill following the same curve. The solution is to keep the system prompt (non-negotiable) and retain only the last N messages.

# BAD — history grows indefinitely
conversation = []
while True:
    user_input = input("You: ")
    conversation.append({"role": "user", "content": user_input})
    response = client.chat.complete(
        model="mistral-large-latest",
        messages=conversation  # 100 turns = thousands of tokens!
    )
    reply = response.choices[0].message.content
    conversation.append({"role": "assistant", "content": reply})

# GOOD — sliding window with trimming
MAX_MESSAGES = 20

def trim_conversation(messages: list, max_messages: int = MAX_MESSAGES) -> list:
    """Keeps the system prompt + the last N messages."""
    system = [m for m in messages if m["role"] == "system"]
    history = [m for m in messages if m["role"] != "system"]

    if len(history) > max_messages:
        history = history[-max_messages:]

    return system + history

Assuming the Network Always Responds

A bare API call assumes nothing will go wrong: no timeouts, no 429s, no 500s. In production, all three happen, and the third happens at the worst possible moment. Note the distinction in the code: a 429 justifies exponential backoff, a server error warrants a second of patience, but a client error like a 400 or 401 must never be retried—your key won’t become valid on the third attempt; you’ll only delay diagnosis.

# BAD — no error handling
response = client.chat.complete(
    model="mistral-large-latest",
    messages=[{"role": "user", "content": "Hello"}]
)
# Crashes on timeout, 429, 500...

# GOOD — retry with backoff
from mistralai.client.errors import SDKError
import time

def safe_complete(client, messages, max_retries=3, **kwargs):
    """API call with retry and exponential backoff."""
    for attempt in range(max_retries):
        try:
            return client.chat.complete(messages=messages, **kwargs)
        except SDKError as e:
            if e.status_code == 429:
                wait = 2 ** attempt
                print(f"Rate limit hit, waiting {wait}s...")
                time.sleep(wait)
            elif e.status_code >= 500:
                time.sleep(1)
            else:
                raise  # Client error (400, 401) — do not retry
    raise Exception("Failed after 3 attempts")

Asking the Model to Do What It Can’t Do

Three anti-patterns converge here. The first is subjective language: “short,” “many,” “interesting” refer to nothing verifiable, and the model will decide differently from one call to the next. Replace every adjective with a number or an observable criterion.

# BAD — subjective, not measurable
"Write a short summary."           # Short = 1 sentence? 1 paragraph?
"Give many examples."              # Many = 3? 10? 50?
"Answer in an interesting way."    # Interesting by what criteria?

# GOOD — objective criteria
"Write a summary of 50 to 80 words."
"Give exactly 5 examples, one per line."
"Start every response with a relevant statistic."

The second is letting the model count. LLMs are notoriously bad at counting words, characters, and items: asking for “exactly 280 characters” will produce text of 240 or 310. Counting belongs in your application, not the model.

# BAD — the model cannot count precisely
"Write exactly 280 characters for a tweet."

# GOOD — count on the application side
response = client.chat.complete(
    model="mistral-large-latest",
    messages=[{"role": "user", "content": "Write a tweet about AI in France."}],
    max_tokens=100
)

tweet = response.choices[0].message.content
if len(tweet) > 280:
    tweet = tweet[:277] + "..."

print(f"Tweet ({len(tweet)} chars): {tweet}")

The third falls into the same family: numerical scales. “Rate from 1 to 5” assumes the model has a stable representation of the gap between 3 and 4, which it doesn’t. A verbal scale where each step is defined by a criterion yields much more reproducible ratings.

# BAD — models interpret numerical scales poorly
"Rate the quality of this text from 1 to 5."

# GOOD — verbal scale with definitions
"""Rate the quality of this text according to the following scale:
- EXCELLENT: Clear, well-structured, no errors
- GOOD: Generally correct, minor improvements possible
- AVERAGE: Understandable but lacks structure or contains errors
- POOR: Difficult to understand, significant errors
- VERY POOR: Incomprehensible or off-topic

Respond only with the label in uppercase."""

Deploying Without Leaving a Trace

The last anti-pattern is the most discreet: calling the API, returning the content, and keeping nothing. The day a user reports an aberrant response, you have neither the request ID, nor the actual model served, nor the token consumption. Structured logging per call costs a few milliseconds and makes investigation possible.

# BAD — no trace
response = client.chat.complete(
    model="mistral-large-latest",
    messages=messages
)
return response.choices[0].message.content

# GOOD — structured logging
import logging
import json

logger = logging.getLogger("mistral_api")

def logged_complete(client, messages, **kwargs):
    """API call with structured logging."""
    response = client.chat.complete(messages=messages, **kwargs)

    logger.info(json.dumps({
        "request_id": response.id,
        "model": response.model,
        "prompt_tokens": response.usage.prompt_tokens,
        "completion_tokens": response.usage.completion_tokens,
        "finish_reason": response.choices[0].finish_reason
    }))

    return response

A Debugging Routine

When a response doesn’t meet your expectations, the temptation is to rewrite the prompt. Instead, start with five mechanical checks, in order: Is the response complete? Did the correct model respond? How many tokens were consumed? Is the response empty? Is the output/input ratio plausible? A ratio below 1% on a large prompt almost always signals that the model refused to process the request rather than mishandling it.

def debug_response(response):
    """Debugging checklist for unexpected responses."""
    choice = response.choices[0]

    # 1. Is the response complete?
    if choice.finish_reason == "length":
        print("ISSUE: Response truncated — increase max_tokens")

    # 2. Was the correct model used?
    print(f"Model: {response.model}")

    # 3. How many tokens consumed?
    print(f"Input tokens: {response.usage.prompt_tokens}")
    print(f"Output tokens: {response.usage.completion_tokens}")

    # 4. Is the response empty?
    if not choice.message.content or not choice.message.content.strip():
        print("ISSUE: Empty response")

    # 5. Suspicious input/output ratio?
    ratio = response.usage.completion_tokens / max(response.usage.prompt_tokens, 1)
    if ratio < 0.01:
        print(f"WARNING: Very low output/input ratio ({ratio:.4f})")

Key Takeaways

• Always check finish_reason to detect truncation
• Limit conversation history with a sliding window
• Implement retry with exponential backoff for network errors
• Use objective, measurable criteria in your prompts
• Never let the model count—validate on the application side
• Log every API call in production for debugging and cost tracking