OpenAI's Moderation API

Updated November 3, 2026 based on OpenAI’s Moderation guide: added the illicit category (and illicit/violent), which was missing from the table, along with categories that apply to images. Source: Moderation.

A Built-In Moderation Tool

OpenAI provides a free Moderation API that analyzes text to detect potentially harmful content. This is the first line of defense to integrate into any application using OpenAI models, and you are not limited to just that use case: since it accepts raw text as input, you can also use it to filter inputs destined for other models. The fact that it is free changes how you think about the problem. The usual trade-offs—should we moderate every request or only suspicious ones?—lose their weight, since the only remaining cost is latency.

What the API Returns

The basic call fits on one line; the response is what demands attention. The API returns three distinct things: a flagged boolean reflecting OpenAI’s decision, the list of categories actually triggered, and, most importantly, continuous scores per category. These scores are what make the difference in production, because they let you decide where to draw the line yourself rather than inheriting a threshold designed for generic use.

from openai import OpenAI

client = OpenAI()

def moderate_content(text: str) -> dict:
    """Analyzes text via OpenAI's Moderation API."""
    response = client.moderations.create(input=text)
    result = response.results[0]

    return {
        "blocked": result.flagged,
        "categories": {
            cat: flagged
            for cat, flagged in result.categories.__dict__.items()
            if flagged
        },
        "scores": {
            cat: round(score, 4)
            for cat, score in result.category_scores.__dict__.items()
            if score > 0.01
        },
    }

# Test with different content types
examples = [
    "How do I bake a chocolate cake?",
    "I will kill you if you don't answer me",
    "Explain how AES encryption works",
]

for text in examples:
    result = moderate_content(text)
    status = "BLOCKED" if result["blocked"] else "OK"
    print(f"{status} : {text[:50]}...")
    if result["categories"]:
        print(f"  Categories : {list(result['categories'].keys())}")

The three test examples are not random: the first should pass, the second should be blocked, and the third—a perfectly legitimate technical question—shows you how your pipeline behaves when faced with security vocabulary. This is exactly the kind of request a cybersecurity team handles daily.

Detected Categories

The API classifies problematic content along several axes, each corresponding to a distinct type of harm.

Each category also has subcategories (e.g., harassment/threatening, illicit/violent, violence/graphic) for finer filtering. With omni-moderation-latest, only self-harm and its variants, sexual, and violence (with violence/graphic) also apply to images; the other categories apply only to text. This granularity saves you from making coarse trade-offs: a forum moderation platform might want to block harassment/threatening while tolerating simple harassment, which covers many ordinary disputes.

When you need to process volume—retrospective analysis of a conversation history, for example—the API accepts a list and returns a result for each element, avoiding multiple network round trips.

def moderate_batch(messages: list[str]) -> list[dict]:
    """Moderates multiple messages in a single call."""
    response = client.moderations.create(input=messages)
    results = []
    for i, result in enumerate(response.results):
        results.append({
            "index": i,
            "text": messages[i][:50],
            "blocked": result.flagged,
            "max_score": max(
                score
                for score in result.category_scores.__dict__.values()
            ),
        })
    return results

Moderating Both Ends of the Pipeline

A common mistake is moderating only inputs. This relies on a false intuition: if the incoming message is acceptable, the response will be too. Yet a model can produce problematic content from a neutral request, especially when it relies on retrieved documents whose content no one has verified. The class below therefore applies the same treatment in both directions, adding a custom threshold that tightens the API’s default decision.

class ModerationPipeline:
    """Input + output moderation pipeline for an LLM application."""

    def __init__(self, custom_threshold: float = 0.7):
        self.client = OpenAI()
        self.threshold = custom_threshold

    def moderate_input(self, message: str) -> tuple[bool, dict]:
        """Moderates the message BEFORE sending to the model."""
        result = self.client.moderations.create(input=message).results[0]
        scores = result.category_scores.__dict__
        high_categories = {
            cat: score for cat, score in scores.items() if score > self.threshold
        }
        blocked = result.flagged or bool(high_categories)
        return blocked, {"flagged_api": result.flagged, "custom_categories": high_categories}

    def moderate_output(self, response: str) -> tuple[bool, dict]:
        """Moderates the model's response BEFORE sending to the user."""
        return self.moderate_input(response)

    def process_message(self, user_message: str, system_prompt: str) -> str:
        """Full pipeline: input moderation, LLM, output moderation."""
        # Step 1: moderate input
        input_blocked, details = self.moderate_input(user_message)
        if input_blocked:
            return "Your message cannot be processed. Please rephrase it."

        # Step 2: call the model
        response = self.client.chat.completions.create(
            model="gpt-5.6-sol",
            messages=[
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": user_message},
            ],
        )
        model_response = response.choices[0].message.content

        # Step 3: moderate output
        output_blocked, _ = self.moderate_output(model_response)
        if output_blocked:
            return "The response was filtered for non-compliance."

        return model_response

# Usage
pipeline = ModerationPipeline(custom_threshold=0.5)
response = pipeline.process_message(
    "How do I protect my application against SQL injections?",
    "You are an expert in defensive cybersecurity."
)
print(response)

What This API Does Not Do

Its most important limitation is also the most misunderstood: it does not detect prompt injection. A polite message that is perfectly harmless from a content perspective, but asks the model to ignore its instructions, will pass without triggering any category. Expecting it to protect against manipulation is like installing a smoke detector and hoping it stops burglaries.

False positives are the second limitation, particularly sensitive in security or medical contexts, where describing a threat is part of the job. An additional network call is added to every request, which matters when you aim for real-time responses. Detection remains more reliable in English than in other languages, a blind spot attackers know perfectly well. Finally, categories and thresholds evolve with updates: a calibration set six months ago deserves re-verification.

These limits dictate how to use the tool. Moderate in both directions, adjust thresholds category by category rather than globally, and systematically combine it with other layers—injection detection, business logic filtering—that the following lessons detail. Log the results, because observing your own false positives is how you learn where to move the sliders. And plan for the case where the API is unavailable: decide now, while things are calm, whether your application should block by default or let traffic through, rather than discovering that behavior during an incident.

Key Takeaways

• The Moderation API is free, fast, and simple to integrate
• It detects offensive content but not injections or jailbreaks
• The input-to-model-to-output pipeline must be moderated at both ends
• Default thresholds rarely fit—adapt them to your context
• It is one layer among others, never the sole line of defense