Content Moderation with the API

Updated November 3, 2026 based on OpenAI’s Moderation guide: the illicit and illicit/violent categories for the omni-moderation-latest model were missing from the list and code. Source: Moderation.

Content Moderation with the API

OpenAI’s Moderation API automatically detects inappropriate content. It is a free, essential tool for any application that accepts user-generated content. The previous lesson showed you how to write custom filters; this one gives you a trained classifier that spots what regular expressions will never catch—veiled threats, indirect insults, and implicit phrasing.

The Moderation API

The call is intentionally minimal: one model, one text, and a result that boils down to a boolean. The flagged field answers the binary question, “Does this content violate usage policies?” while categories details which ones. Since this endpoint is free, there is no economic reason to skip it, even at high volumes.

from openai import OpenAI
client = OpenAI()

# The Moderation API is FREE
moderation = client.moderations.create(
    model="omni-moderation-latest",
    input="Hello, how are you?"
)

result = moderation.results[0]
print(f"Content flagged: {result.flagged}")
print(f"Categories: {result.categories}")

# Result:
# Content flagged: False
# Categories: Categories(harassment=False, hate=False, ...)

Moderation Categories

Behind the boolean lies a fine-grained grid. The model returns a decision per category—harassment, threatening harassment, hate, threatening hate, instructions for illicit acts (with or without violence), self-harm with its variants (instructions and intent), sexual content, sexual content involving minors, violence, and graphic violence—along with a confidence score between 0.0 and 1.0 for each.

These scores matter more than the binary decision when you want to calibrate your policy. A professional forum platform might want to block at 0.3 for harassment, whereas a fiction service might accept 0.7 for violence. The function below exposes both levels of insight: the categories actually triggered, and the raw scores for your own thresholds.

def analyze_moderation(text: str) -> dict:
    """Detailed moderation analysis."""
    moderation = client.moderations.create(
        model="omni-moderation-latest",
        input=text
    )

    result = moderation.results[0]

    # Available categories
    categories = {
        "harassment": result.categories.harassment,
        "harassment_threatening": result.categories.harassment_threatening,
        "hate": result.categories.hate,
        "hate_threatening": result.categories.hate_threatening,
        "illicit": result.categories.illicit,
        "illicit_violent": result.categories.illicit_violent,
        "self_harm": result.categories.self_harm,
        "self_harm_instructions": result.categories.self_harm_instructions,
        "self_harm_intent": result.categories.self_harm_intent,
        "sexual": result.categories.sexual,
        "sexual_minors": result.categories.sexual_minors,
        "violence": result.categories.violence,
        "violence_graphic": result.categories.violence_graphic,
    }

    # Confidence scores (0.0 to 1.0)
    scores = {
        "harassment": result.category_scores.harassment,
        "hate": result.category_scores.hate,
        "self_harm": result.category_scores.self_harm,
        "sexual": result.category_scores.sexual,
        "violence": result.category_scores.violence,
    }

    return {
        "flagged": result.flagged,
        "detected_categories": [k for k, v in categories.items() if v],
        "main_scores": {k: round(v, 4) for k, v in scores.items()},
    }

# Test
result = analyze_moderation("Hello everyone!")
print(f"Flagged: {result['flagged']}")
print(f"Scores: {result['main_scores']}")

Complete Moderation Pipeline

Moderating User Inputs

Let’s wrap this call in a reusable component. ContentModerator returns an explicit pair—acceptable or not, with the reason—rather than a raw object that every caller would have to reinterpret. Translating categories into clear language matters: this is the vocabulary that will appear in your interface and logs, and it must be understandable by the human moderation team that takes over.

class ContentModerator:
    """Content moderator for applications."""

    def __init__(self, custom_threshold: float = 0.5):
        self.client = OpenAI()
        self.threshold = custom_threshold

    def moderate(self, text: str) -> tuple[bool, dict]:
        """Moderates text. Returns (is_acceptable, details)."""
        moderation = self.client.moderations.create(
            model="omni-moderation-latest",
            input=text
        )

        result = moderation.results[0]

        if result.flagged:
            categories = []
            if result.categories.harassment:
                categories.append("harassment")
            if result.categories.hate:
                categories.append("hate speech")
            if result.categories.sexual:
                categories.append("sexual content")
            if result.categories.violence:
                categories.append("violence")
            if result.categories.self_harm:
                categories.append("self-harm")

            return False, {
                "reason": "Inappropriate content detected",
                "categories": categories
            }

        return True, {"reason": "Content acceptable"}

    def moderate_conversation(self, messages: list[str]) -> list[dict]:
        """Moderates a list of messages."""
        results = []
        for msg in messages:
            acceptable, details = self.moderate(msg)
            results.append({
                "message": msg[:50] + "..." if len(msg) > 50 else msg,
                "acceptable": acceptable,
                **details
            })
        return results

# Usage
moderator = ContentModerator()
acceptable, details = moderator.moderate("Hello, how can I help you?")
print(f"Acceptable: {acceptable}")
print(f"Details: {details}")

Integration into the API Pipeline

Moderation frames the call on both sides. Filtering input avoids paying for generation on content you would reject anyway; filtering output protects against cases where the model, skillfully provoked, produces something unacceptable itself. Note the difference in handling: for input, you can tell the user what was detected, since they authored it; for output, the message stays vague, because detailing it would mean describing what the model just produced.

def moderated_chatbot(user_input: str) -> str:
    """Chatbot with moderation of both inputs AND outputs."""
    moderator = ContentModerator()

    # 1. Moderate input
    input_ok, input_details = moderator.moderate(user_input)
    if not input_ok:
        return (f"Your message was blocked: "
                f"{', '.join(input_details['categories'])}")

    # 2. Generate response
    response = client.responses.create(
        model="gpt-5.6-terra",
        instructions="Respond in a professional and safe manner.",
        input=user_input
    )

    # 3. Moderate output
    output_ok, output_details = moderator.moderate(response.output_text)
    if not output_ok:
        return "I am unable to provide that response."

    return response.output_text

# Test
print(moderated_chatbot("What is the capital of France?"))

Batch Moderation

This double call adds latency, which is acceptable in conversation but becomes prohibitive when you need to process ten thousand customer reviews imported from a file. To moderate large volumes efficiently, pass an array directly: the API processes all texts in a single request and returns results in the same order, allowing you to reassociate them by index.

def moderate_batch(texts: list[str]) -> list[dict]:
    """Moderates multiple texts in a single request."""
    results = []

    # The API accepts an array of inputs
    moderation = client.moderations.create(
        model="omni-moderation-latest",
        input=texts
    )

    for i, result in enumerate(moderation.results):
        results.append({
            "index": i,
            "text_preview": texts[i][:50],
            "flagged": result.flagged,
        })

    flagged_count = sum(1 for r in results if r["flagged"])
    print(f"Moderated {len(texts)} texts: {flagged_count} flagged")

    return results

# Usage
texts = [
    "Hello, how are you?",
    "Great product, I recommend it!",
    "Customer service is excellent.",
]
results = moderate_batch(texts)

Moderation Logging

A moderation decision affects a user, and you will sooner or later need to justify it—whether to a customer contesting a block or to a regulator. Therefore, systematically log the verdict, the user ID, the scores, and a text excerpt. The log level distinguishes the two cases: warning for flagged content, which merits human review, and info for the rest, which feeds your statistics.

import json
import logging
from datetime import datetime

logger = logging.getLogger("moderation")

def moderate_with_log(text: str, user_id: str = "anonymous") -> bool:
    """Moderates and logs the result for auditing."""
    moderation = client.moderations.create(
        model="omni-moderation-latest",
        input=text
    )

    result = moderation.results[0]

    log_entry = {
        "timestamp": datetime.now().isoformat(),
        "user_id": user_id,
        "flagged": result.flagged,
        "text_preview": text[:100],
        "scores": {
            "harassment": round(result.category_scores.harassment, 4),
            "hate": round(result.category_scores.hate, 4),
            "violence": round(result.category_scores.violence, 4),
        }
    }

    if result.flagged:
        logger.warning(json.dumps(log_entry))
    else:
        logger.info(json.dumps(log_entry))

    return not result.flagged

# Usage
is_ok = moderate_with_log("Test message", user_id="user-123")

One final warning before moving on: moderation classifies content; it does not understand intent. A perfectly polite message attempting to jailbreak your assistant will pass all these filters without being flagged once. That is the subject of the next lesson.

Key Takeaways

• The Moderation API is free—use it systematically
• Moderate both inputs (user content) AND outputs (model responses)
• The omni-moderation-latest model covers all content categories
• Log all moderation decisions for auditing
• Use batch processing to moderate large volumes efficiently
• Moderation complements system instructions; it does not replace them