Structured Entity Extraction
On this page
Lesson updated November 3, 2026 based on OpenAI documentation:
temperatureandtop_pare only accepted with a reasoning effort ofnone, whereas GPT-5.6 models reason by default (medium). Calls that set these parameters must therefore specifyreasoning={"effort": "none"}(Responses API) orreasoning_effort="none"(Chat Completions). Sources: Using GPT-6 · GPT-5.6 Terra.
Structured Entity Extraction
Structured entity extraction is one of the most requested use cases in production: transforming free text (emails, documents, web pages) into data your application can actually use. By combining Structured Output with specialized prompts, you can build reliable extraction pipelines without training a dedicated NER model.
Basic Extraction Pipeline
The entire specificity of extraction lies in the system prompt of the function below, and it deserves close attention. Three instructions overlap here: extract only what is present, use null for missing data, and never infer information that isn’t explicitly mentioned.
That last sentence is the most important in practice. Left to its own devices, a model will happily fill in a plausible SIRET number or guess a city from an area code, with the best of intentions. For extraction destined for a business system, this helpfulness is a serious flaw: once the data is in the database, there is no way to distinguish what was read from what was assumed. Setting temperature to 0.0 serves the same purpose—across two runs of the same document, you want the same output, if only to reproduce an incident.
import json
from openai import OpenAI
client = OpenAI()
def extract_entities(text: str, schema: dict,
instructions: str = "") -> dict:
"""Extracts structured entities from free text."""
system = (
"You are a specialized entity extractor. "
"Extract ONLY information present in the text. "
"If information is missing, use null. "
"NEVER infer information not explicitly mentioned."
)
if instructions:
system += f"\n\nAdditional instructions: {instructions}"
response = client.responses.create(
model="gpt-5.6-terra",
reasoning={"effort": "none"}, # temperature requires effort none
instructions=system,
input=f"Text to analyze:\n\n{text}",
text={
"format": {
"type": "json_schema",
"name": "extraction",
"schema": schema,
"strict": True
}
},
temperature=0.0
)
return json.loads(response.output_text)Example 1: Resume Extraction
A resume is an excellent testing ground because it combines stable sections with completely free-form layout. The schema below groups contact details into an identity object where almost every field is nullable—a candidate might not provide a phone number, and making that field required would essentially invite the model to invent one.
Skills are not simple strings but objects with three attributes, which allows you to preserve the stated level and categorize each skill. Without these enums, “Scrum” and “Python” would end up side by side in the same flat list, useless for a filter trying to isolate languages. Also note the end_date field in experiences, which is nullable whereas start_date is not: this is how you encode a current position without inventing a homegrown convention like “present” or “9999”. On the test resume, this subtlety plays out on the TechCorp line.
resume_schema = {
"type": "object",
"properties": {
"identity": {
"type": "object",
"properties": {
"full_name": {"type": "string"},
"email": {"type": ["string", "null"]},
"phone": {"type": ["string", "null"]},
"location": {"type": ["string", "null"]},
"linkedin": {"type": ["string", "null"]}
},
"required": ["full_name", "email", "phone",
"location", "linkedin"],
"additionalProperties": False
},
"current_title": {"type": ["string", "null"]},
"years_experience": {"type": ["integer", "null"]},
"skills": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {"type": "string"},
"level": {
"type": "string",
"enum": ["beginner", "intermediate",
"advanced", "expert"]
},
"category": {
"type": "string",
"enum": ["language", "framework", "tool",
"methodology", "soft_skill"]
}
},
"required": ["name", "level", "category"],
"additionalProperties": False
}
},
"experiences": {
"type": "array",
"items": {
"type": "object",
"properties": {
"company": {"type": "string"},
"title": {"type": "string"},
"start_date": {"type": "string"},
"end_date": {"type": ["string", "null"]},
"description": {"type": "string"}
},
"required": ["company", "title", "start_date",
"end_date", "description"],
"additionalProperties": False
}
},
"education": {
"type": "array",
"items": {
"type": "object",
"properties": {
"institution": {"type": "string"},
"degree": {"type": "string"},
"year": {"type": ["string", "null"]}
},
"required": ["institution", "degree", "year"],
"additionalProperties": False
}
},
"languages": {
"type": "array",
"items": {
"type": "object",
"properties": {
"language": {"type": "string"},
"level": {
"type": "string",
"enum": ["a1", "a2", "b1", "b2",
"c1", "c2", "native"]
}
},
"required": ["language", "level"],
"additionalProperties": False
}
}
},
"required": ["identity", "current_title", "years_experience",
"skills", "experiences", "education", "languages"],
"additionalProperties": False
}
resume_text = """
Marie Dupont - Senior Full Stack Developer
Email: [email protected] | Tel: +33 6 12 34 56 78
Paris, France | linkedin.com/in/mariedupont
8 years of experience in web development.
Skills: Python (expert), TypeScript (advanced), React (advanced),
Django (expert), PostgreSQL (advanced), Docker (intermediate),
Scrum (advanced)
Experience:
- TechCorp (2022-present): Lead Full Stack Developer
Principal architect of a B2B SaaS platform, team of 6 devs.
- StartupAI (2019-2022): Senior Developer
Development of ML APIs in Python/FastAPI.
Education:
- EPITECH Paris, Master in Computer Science (2017)
Languages: French (native), English (C1), Spanish (B1)
"""
result = extract_entities(resume_text, resume_schema)Example 2: Commercial Email Extraction
The second example changes nature: we are no longer just copying information, we are asking the model to interpret. intent and urgency are judgments—no sentence in the email says “this is a quote request, high priority.” This is precisely what a classic extraction pipeline based on rules or regular expressions cannot do, and it is exactly what justifies using an LLM here rather than elsewhere.
The rest of the schema returns to verifiable facts: mentioned products, amounts with their context, dates and their significance, required actions. The summary field closes the object and serves the human who will review the queue, not the code. One point of vigilance with this mixed schema: a judgment error on urgency does not invalidate the amounts, but the reverse can happen, because a model that misunderstands the intent sometimes reads the rest through that misconception. If classification is critical for your routing, separate it into a dedicated call.
email_schema = {
"type": "object",
"properties": {
"intent": {
"type": "string",
"enum": ["quote_request", "complaint", "information",
"order", "partnership", "other"]
},
"urgency": {
"type": "string",
"enum": ["critical", "high", "normal", "low"]
},
"sender": {
"type": "object",
"properties": {
"name": {"type": ["string", "null"]},
"company": {"type": ["string", "null"]},
"email": {"type": ["string", "null"]}
},
"required": ["name", "company", "email"],
"additionalProperties": False
},
"mentioned_products": {
"type": "array",
"items": {"type": "string"}
},
"mentioned_amounts": {
"type": "array",
"items": {
"type": "object",
"properties": {
"value": {"type": "number"},
"currency": {"type": "string"},
"context": {"type": "string"}
},
"required": ["value", "currency", "context"],
"additionalProperties": False
}
},
"key_dates": {
"type": "array",
"items": {
"type": "object",
"properties": {
"date": {"type": "string"},
"significance": {"type": "string"}
},
"required": ["date", "significance"],
"additionalProperties": False
}
},
"required_actions": {
"type": "array",
"items": {"type": "string"}
},
"summary": {"type": "string"}
},
"required": ["intent", "urgency", "sender",
"mentioned_products", "mentioned_amounts",
"key_dates", "required_actions", "summary"],
"additionalProperties": False
}Batch Extraction
To process large volumes, parallelize your extractions. A sequential loop over a thousand documents spends most of its time waiting on the network; with asyncio, these waits overlap and the batch processes in a fraction of the time.
The semaphore is not a decorative detail. Without it, launching a thousand tasks at once will saturate your quota and trigger rate limit errors that cost more in retries than they save in speed. Set the max_concurrent value by observing your own limits, with ten being a cautious starting point. Also note return_exceptions=True: it prevents a single malformed document from crashing the entire batch, leaving you to sort out failures afterward.
import asyncio
from openai import AsyncOpenAI
async_client = AsyncOpenAI()
async def extract_batch(texts: list[str], schema: dict,
max_concurrent: int = 10) -> list[dict]:
"""Parallel extraction with concurrency control."""
semaphore = asyncio.Semaphore(max_concurrent)
async def extract_one(text: str) -> dict:
async with semaphore:
response = await async_client.responses.create(
model="gpt-5.6-terra",
reasoning={"effort": "none"}, # temperature requires effort none
instructions="Extract entities from the provided text. "
"Use null for missing fields.",
input=text,
text={
"format": {
"type": "json_schema",
"name": "extraction",
"schema": schema,
"strict": True
}
},
temperature=0.0
)
return json.loads(response.output_text)
tasks = [extract_one(t) for t in texts]
return await asyncio.gather(*tasks, return_exceptions=True)Evaluating Extraction Quality
An extraction pipeline without an annotated test set is a pipeline whose reliability no one knows, and the impression left by three examples read by hand almost always proves too optimistic. The function below compares an obtained extraction field-by-field against a manually established reference, returning overall precision along with a breakdown of discrepancies.
This breakdown is the useful part: errors are almost never distributed uniformly. One field concentrates the problems—often an ambiguous date, an implicit skill level, or a poorly defined category—and a single sentence added to the system prompt fixes it. Without this measurement, you would be tweaking the prompt at random, possibly worsening a field that was already working.
def evaluate_extraction(expected: dict, obtained: dict,
fields: list[str]) -> dict:
"""Compares obtained extraction with the reference."""
correct = 0
total = len(fields)
errors = []
for field in fields:
val_expected = expected.get(field)
val_obtained = obtained.get(field)
if val_expected == val_obtained:
correct += 1
else:
errors.append({
"field": field,
"expected": val_expected,
"obtained": val_obtained
})
return {
"precision": correct / total if total > 0 else 0,
"correct": correct,
"total": total,
"errors": errors
}Build Your Reference Before Optimizing
Choose a document type from your domain—contracts, invoices, reports—and define an extraction schema of at least ten fields, deliberately mixing easy fields with ones you know are ambiguous. Assemble five example documents with their expected extractions, written by hand: this step is tedious, it takes an hour, and it is what gives all subsequent work its value.
Then run the pipeline, measure precision, and read the list of errors field by field. The most faulty field will tell you what to correct in the prompt; a second measurement will tell you if the correction actually helped—or if it degraded another field in the process, which happens more often than you might think as soon as an additional instruction shifts the model’s attention.
Key Takeaways
- Structured extraction replaces dedicated NER models for most use cases
- Use
temperature=0.0to maximize consistency - Make optional fields nullable rather than optional
- Parallelize with asyncio for large volumes
- Systematically evaluate quality with an annotated test set
- System prompt instructions guide content, schema constrains structure