Security audit and certification

From ad hoc security to certifiable compliance

The previous lessons gave you the technical tools to secure an AI system. But one-off security checks aren’t enough—and the reason isn’t just bureaucracy. What you verify once stops being verified as soon as you ship the next release, and nothing flags that gap. Customers, regulators, and partners demand formal evidence: proof that your controls exist, work, and hold up over time. Audit and certification turn your practices into a verifiable, continuous framework.

Know which framework applies

Five frameworks intersect in AI, each with a different scope. Distinguish them before you commit to anything.

The decisive distinction separates what’s mandatory from what’s optional. The EU AI Act is a regulation: if your system falls into the high-risk category, you don’t get to opt out. ISO standards and SOC 2 are voluntary certifications, typically demanded by enterprise customers rather than regulators. The OWASP LLM Top 10 isn’t certifiable—it’s a technical reference, and it’s the best starting point for a first internal audit because it costs only time.

A checklist that covers five domains

A serious audit goes beyond technique. The checklist below spreads its control points across five domains, and that distribution tells a story: governance and compliance weigh as much as security, because a flawless technical setup that no one owns and nowhere documents will fail any external audit.

Criticality separates mandatory from recommended, a distinction that later drives a binary decision. Sandboxing appears as recommended rather than mandatory, and that ranking is justified: it applies only to systems that execute generated code, whereas input and output filtering concerns every application without exception. A well-calibrated checklist doesn’t ask a FAQ chatbot to justify container isolation.

from dataclasses import dataclass, field
from datetime import datetime

@dataclass
class ControlPoint:
    id: str
    category: str
    description: str
    criticality: str  # mandatory, recommended, optional
    compliant: bool | None = None
    evidence: str = ""
    comment: str = ""

def create_audit_checklist() -> list[ControlPoint]:
    """Creates a comprehensive AI security audit checklist."""
    return [
        # Governance
        ControlPoint("GOV-01", "Governance",
            "Documented and approved AI security policy", "mandatory"),
        ControlPoint("GOV-02", "Governance",
            "Designated AI security owner (CISO/DPO)", "mandatory"),
        ControlPoint("GOV-03", "Governance",
            "Up-to-date register of AI processing activities", "mandatory"),

        # Data
        ControlPoint("DAT-01", "Data",
            "Inventory of data sent to LLM APIs", "mandatory"),
        ControlPoint("DAT-02", "Data",
            "Anonymization/pseudonymization of personal data", "mandatory"),
        ControlPoint("DAT-03", "Data",
            "DPA signed with each AI API provider", "mandatory"),
        ControlPoint("DAT-04", "Data",
            "Defined conversation retention policy", "mandatory"),

        # Technical security
        ControlPoint("SEC-01", "Security",
            "Prompt injection: defense-in-depth implemented", "mandatory"),
        ControlPoint("SEC-02", "Security",
            "Content filtering (inputs and outputs)", "mandatory"),
        ControlPoint("SEC-03", "Security",
            "Rate limiting and abuse detection", "mandatory"),
        ControlPoint("SEC-04", "Security",
            "API keys stored in a vault (not hardcoded)", "mandatory"),
        ControlPoint("SEC-05", "Security",
            "Sandboxing for generated code execution", "recommended"),
        ControlPoint("SEC-06", "Security",
            "Monitoring and alerts on anomalies", "recommended"),

        # Red teaming
        ControlPoint("RED-01", "Red teaming",
            "Security tests before every production release", "mandatory"),
        ControlPoint("RED-02", "Red teaming",
            "Regular red teaming (quarterly minimum)", "recommended"),
        ControlPoint("RED-03", "Red teaming",
            "Red team report with remediation tracking", "mandatory"),

        # Compliance
        ControlPoint("CONF-01", "Compliance",
            "DPIA completed for AI processing activities", "mandatory"),
        ControlPoint("CONF-02", "Compliance",
            "Data subject rights implemented (access, erasure)", "mandatory"),
        ControlPoint("CONF-03", "Compliance",
            "Transparent user notice about AI usage", "mandatory"),
        ControlPoint("CONF-04", "Compliance",
            "Risk classification per the EU AI Act", "recommended"),
    ]

Audit means demanding evidence

The field that changes everything in audit execution is evidence. Checking “compliant” without a document reference means nothing to an external auditor, who will systematically ask what supports the claim. The usage example shows this clearly: GOV-01 compliance rests on a named, dated document, not a statement of intent.

The third call illustrates the most instructive case—a non-compliance paired with its remediation plan. An API key in an environment file isn’t a scandal; it’s a common practice. It still fails the control point, and the comment lays out the path to compliance. An audit that surfaces no non-compliances should raise suspicion: it usually signals a lenient checklist, not a perfect system.

The final decision is binary and unweighted: a single mandatory non-compliance flips the verdict. The compliance rate, calculated alongside, measures progress between audits—never negotiates the verdict itself.

class AISecurityAudit:
    """Executes and documents an AI security audit."""

    def __init__(self, system_name: str, auditor: str):
        self.system_name = system_name
        self.auditor = auditor
        self.date = datetime.now().strftime("%Y-%m-%d")
        self.checklist = create_audit_checklist()

    def evaluate(self, point_id: str, compliant: bool, evidence: str, comment: str = ""):
        """Evaluates a control point."""
        for point in self.checklist:
            if point.id == point_id:
                point.compliant = compliant
                point.evidence = evidence
                point.comment = comment
                return
        raise ValueError(f"Unknown control point: {point_id}")

    def report(self) -> dict:
        """Generates the audit report."""
        evaluated = [p for p in self.checklist if p.compliant is not None]
        compliant = [p for p in evaluated if p.compliant]
        non_compliant = [p for p in evaluated if not p.compliant]
        not_evaluated = [p for p in self.checklist if p.compliant is None]

        # Mandatory non-compliances are blockers
        blockers = [
            p for p in non_compliant if p.criticality == "mandatory"
        ]

        return {
            "system": self.system_name,
            "auditor": self.auditor,
            "date": self.date,
            "total_controls": len(self.checklist),
            "evaluated": len(evaluated),
            "compliant": len(compliant),
            "non_compliant": len(non_compliant),
            "not_evaluated": len(not_evaluated),
            "blockers": len(blockers),
            "compliance_rate": f"{len(compliant) / max(len(evaluated), 1) * 100:.0f}%",
            "decision": "NON-COMPLIANT" if blockers else "COMPLIANT",
            "blocking_non_compliances": [
                {"id": p.id, "description": p.description, "comment": p.comment}
                for p in blockers
            ],
        }

# Usage example
audit = AISecurityAudit("Customer Support Chatbot v2.3", "security_analyst")
audit.evaluate("GOV-01", True, "Document POL-SEC-AI-2026 approved on 2026-01-15")
audit.evaluate("SEC-01", True, "Multi-layer guardrails implemented (see architecture)")
audit.evaluate("SEC-04", False, "OpenAI API key in .env environment variable",
              "Migrate to a vault (HashiCorp Vault or AWS Secrets Manager)")

result = audit.report()
print(f"Decision: {result['decision']}")
print(f"Compliance rate: {result['compliance_rate']}")

One last counter—the number of unevaluated points—plays a quiet but decisive role. It prevents you from confusing “compliant” with “not checked,” a confusion that creeps in when an audit stretches over weeks and no one remembers where it stopped.

Automate what you can

An annual audit captures a snapshot. Between snapshots, an API key can get hardcoded, a rate limit disabled to unblock a rushed customer, or moderation bypassed for a quick test. Automated checks fill that gap by running on every continuous integration cycle.

Not everything lends itself to automation, and you should say so plainly. Searching for hardcoded keys is mechanizable because it inspects code. The other two checks in the example return hardcoded values, which only makes sense as a starting point: in production, they must actually exercise the service—call the API twenty-one times to confirm a 429 appears, or submit test content to moderation to confirm it responds. A check that always returns true is worse than no check, because it manufactures confidence without measuring anything.

class ContinuousAudit:
    """Automatically verifies security controls on an ongoing basis."""

    def __init__(self):
        self.checks = {}

    def add_check(self, name: str, check_function):
        """Adds an automated check."""
        self.checks[name] = check_function

    def run(self) -> dict:
        """Runs all checks."""
        results = {}
        for name, check in self.checks.items():
            try:
                ok, detail = check()
                results[name] = {"status": "PASS" if ok else "FAIL", "detail": detail}
            except Exception as e:
                results[name] = {"status": "ERROR", "detail": str(e)}
        return results

# Automated checks
continuous_audit = ContinuousAudit()

def check_api_keys():
    """Verifies that no API keys are hardcoded in the codebase."""
    import subprocess
    result = subprocess.run(
        ["grep", "-r", "sk-", "src/", "--include=*.py", "-l"],
        capture_output=True, text=True,
    )
    files = result.stdout.strip().split("\n") if result.stdout.strip() else []
    return len(files) == 0, f"{len(files)} file(s) with hardcoded key"

def check_rate_limiting():
    """Verifies that rate limiting is active."""
    return True, "Rate limiting active (429 after 20 req/min)"

def check_moderation():
    """Verifies that moderation is active on the pipeline."""
    return True, "Moderation API called on inputs and outputs"

continuous_audit.add_check("api_keys_secured", check_api_keys)
continuous_audit.add_check("rate_limiting_active", check_rate_limiting)
continuous_audit.add_check("moderation_active", check_moderation)

What a certifier expects

Aiming for a certification like ISO 42001 or SOC 2 shifts the effort, and you should know that before you commit. Documentation comes first: up-to-date policies, procedures, and registers—the work engineering teams most readily defer. Evidence follows—logs, test reports, screenshots—and is far easier to collect if you archive it as you go rather than three weeks before the audit. History is the hardest requirement to catch up on: a SOC 2 Type II examines how controls operated over several months, something no last-minute push can fabricate. Training comes next, and you must prove it happened, which means dated attendance sheets and materials. Finally, continuous improvement requires showing that findings get fixed and retested—exactly the revalidation loop from the previous chapter, whose value you now see: it produces the evidence the certifier will demand.

The audit, fix, retest, certify cycle has no end. Every new feature reopens control points, and that’s the very meaning of “continuous” in “continuous improvement.”

Key takeaways

• Audit turns security best practices into verifiable evidence
• The checklist covers governance, data, technical security, red teaming, and compliance
• Mandatory non-compliances are blockers—they must be fixed before deployment
• Continuous audit automates repetitive checks (API keys, rate limiting, moderation)
• Certification (ISO 42001, SOC 2) requires documentation, evidence, and history: the audit, fix, retest cycle is continuous, never one-off

Test your knowledge

Offensive and defensive security: the final control before the audit.

1. What is a prompt injection attack?

Answer: Malicious instructions slipped into the data the model processes (web page, document, message) to hijack its behavior—defense: isolate instructions from data, validate outputs, limit exposed capabilities.

2. Jailbreaking vs. prompt injection: what's the difference?

Answer: Jailbreaking tries to make the model bypass its rules through direct conversation; injection works through third-party content it processes. Both are tested—and countered—differently.

3. Why multi-layer filtering?

Answer: Because no single layer is perfect: input moderation, application guardrails, output controls, and rate limiting complement each other—defense in depth applied to AI.

4. What does an AI red teaming campaign involve?

Answer: Methodically attacking your own system (injection scenarios, jailbreaks, exfiltration) with a risk taxonomy, documenting breaches, prioritizing remediation—then retesting.

5. What does the GDPR angle cover in an OpenAI deployment?

Answer: The legal basis for processing, data minimization for data sent, data subject rights, and contractual safeguards—compliance is designed with the architecture, not after.

A safe AI system is proven: taxonomy, offensive tests, layered defenses, audit—the cycle this course had you practice.