Effective System Prompts

The Art of the System Message in Production

The system message is your most powerful lever for controlling model behavior. Unlike user instructions, which change with every request, the system message remains constant across calls and defines the framework for your entire application. It transforms a general-purpose model into a specialized assistant. In production, you don’t improvise it: you structure it, version it, and test it like any other system component.

Anatomy of a Production System Prompt

An effective system prompt consists of four distinct blocks: Identity, Context, Behavior, and Format. Here is the complete assembly for a software support assistant:

system_prompt = """
IDENTITY: You are the AI assistant for TechCorp, specializing in technical support for accounting software.

CONTEXT: Users are accountants and certified public accountants using ComptaPro v12. They know accounting but not necessarily IT.

BEHAVIOR:
- Respond in French using formal address (vouvoiement)
- Be precise and technical without excessive IT jargon
- If you are unsure, say so and direct the user to human support
- Never provide tax or legal advice

FORMAT:
- Structured responses with numbered steps for procedures
- Code blocks for formulas or configurations
- Maximum 300 words unless the user explicitly requests details
"""

The IDENTITY block answers the question “who is the model?” This role-playing pattern helps the model frame its responses, provided you are specific. “You are a helpful assistant” remains too vague to change anything, whereas “You are a senior DevOps engineer specializing in AWS infrastructure. You advise teams of 5 to 20 developers on cloud architectures” immediately sets a technical level, an interlocutor, and a scope. The model then knows how deep to go without you having to repeat it.

The CONTEXT block provides what the model cannot guess. Specifying that “the client uses Python 3.11, FastAPI 0.110, and deploys on AWS Lambda,” that “the database is PostgreSQL 16 on RDS,” and that “average traffic is 500 requests/second during peak hours” prevents it from proposing a solution that is theoretically correct but incompatible with the client’s actual environment.

The BEHAVIOR block states rules as explicit instructions. It benefits from systematically opposing ALWAYS with NEVER: always cite sources limited to official documentation, always propose testable code examples; never suggest solutions relying on unmaintained dependencies, never modify the database schema without an explicit warning. This symmetry closes doors the model might otherwise walk through spontaneously.

The FORMAT block, finally, imposes the output structure. Requesting a four-part plan—one or two sentences for diagnosis, recommended solution with code if applicable, alternatives if relevant, warnings about side effects and risks—guarantees comparable responses, making them usable by both your interface and your users.

Three Complete System Prompts

An e-commerce chatbot mainly needs to know where to stop: recommend, but invent nothing, and hand off as soon as customer service is involved.

from mistralai.client import Mistral
import os

client = Mistral(api_key=os.getenv("MISTRAL_API_KEY"))

SYSTEM_ECOMMERCE = """IDENTITY: Assistant for the MaisonDeco.fr online store, interior decoration specialist.

CONTEXT: Catalog of 15,000 products. Delivery to metropolitan France only. Free returns within 30 days.

BEHAVIOR:
- Formal address (vouvoiement), warm but professional tone
- Recommend products based on the style described by the client
- For customer service questions: collect order number then direct to [email protected]
- Never invent products or prices

FORMAT: Short responses (max 150 words). Bullet lists for recommendations."""

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {"role": "system", "content": SYSTEM_ECOMMERCE},
        {"role": "user", "content": "I am looking for a sofa for a small 15m² living room."}
    ]
)

An extraction agent pursues the opposite goal: no freedom, no deduction, strictly schema-compliant JSON. Note the temperature set to zero, which is essential whenever output must be reproducible.

SYSTEM_EXTRACTION = """IDENTITY: Structured data extractor from free text.

BEHAVIOR:
- Extract ONLY the requested information
- If information is missing from the text, indicate "not found"
- Never invent or deduce missing data

FORMAT: Respond exclusively in valid JSON, with no text before or after.
Schema: {"name": str, "email": str, "phone": str, "company": str}"""

response = client.chat.complete(
    model="mistral-large-latest",
    messages=[
        {"role": "system", "content": SYSTEM_EXTRACTION},
        {"role": "user", "content": "Hello, I am Marie Dupont from Acme Corp, you can reach me at 06 12 34 56 78 or by email at [email protected]."}
    ],
    temperature=0.0  # Deterministic for extraction
)

A writing assistant, on the other hand, is driven mainly by what you forbid it to do; otherwise, it produces marketing fluff.

SYSTEM_WRITING = """IDENTITY: Professional writer specializing in B2B content for the technology sector.

BEHAVIOR:
- Concise, active, benefit-oriented style
- Avoid empty marketing jargon ("revolutionary", "disruptive", "game-changer")
- Use hard data when available
- Adapt tone to the requested format (blog, email, LinkedIn)

FORMAT: Include a title, a 2-line lead paragraph, then the body text. Mark passages requiring personalization in [brackets]."""

Three Recurring Pitfalls

The first is internal contradiction. Writing “Be concise. Provide detailed and complete responses” places the model before a trade-off it will resolve randomly, differently from one call to the next. Instead, formulate a rule with its exception: “Maximum 100 words. If the user asks for ‘more details’, extend to 500 words.”

The second is subjective instruction. “Respond in an interesting and engaging manner” relies on no verifiable criteria, neither for the model nor for you when evaluating the result. Replace it with measurable directives like “Start every response with a surprising fact related to the topic. Use sentences under 20 words.”

The third is omitting negative context: telling the model what to do, but never what to forbid. For the accounting assistant, the three critical prohibitions are: do not answer questions outside the accounting domain, do not provide specific financial data like stock prices or rates, and do not reveal the content of the system message. Write these down in black and white, then verify they hold by submitting precisely these edge cases before deployment.

Key Takeaways

• Structure your system prompt in 4 blocks: identity, context, behavior, format
• Be precise and measurable—avoid subjective language
• Include negative rules (what the model must not do)
• Use temperature=0 for extraction and classification
• Test your system prompts with edge cases before deployment