Context strategies for agents
On this page
The context challenge for agents
An autonomous agent chains together dozens of API calls: it reasons, uses tools, reads results, and iterates. At each step, the context grows. The difference from an ordinary conversation is that this growth doesn’t come from the user but from the agent itself: a SQL query returning three hundred rows, a full web page retrieved, or an entire log file read. Without a management strategy, an agent hits the context limit within a few iterations, often before finishing the task you assigned it.
Designing an agent that manages its memory effectively therefore comes down to answering three questions: what must remain intact, what can be summarized, and what should never have entered the context in the first place. We’ll address them in that order.
A three-level memory
The structure we use draws on the classic distinction between working memory, episodic memory, and semantic memory. Working memory holds the most recent actions in their raw form because the agent needs them exactly as they are to decide what to do next. Episodic memory keeps summaries of past steps: the agent knows it explored a certain lead and what that yielded, without rereading the details. Finally, semantic memory accumulates facts established once and for all—the customer ID, the database schema version, the budget constraint—that must never get lost in a summary.
from dataclasses import dataclass, field
@dataclass
class AgentMemory:
"""Three-level memory system for an agent."""
# Level 1: working memory (current context)
working: list[dict] = field(default_factory=list)
# Level 2: episodic memory (summaries of past steps)
episodes: list[str] = field(default_factory=list)
# Level 3: semantic memory (permanent facts)
facts: list[str] = field(default_factory=list)
max_tokens_working: int = 20_000
max_episodes: int = 10
def add_action(self, action: str, result: str):
"""Adds an action to working memory."""
self.working.append({
"action": action,
"result": result[:2000], # Truncate long results
})
self._check_compaction()
def add_fact(self, fact: str):
"""Adds a permanent fact."""
if fact not in self.facts:
self.facts.append(fact)
def _check_compaction(self):
tokens = sum(
len(str(e)) // 4 for e in self.working
)
if tokens > self.max_tokens_working:
self._compact()
def _compact(self):
"""Compacts working memory into an episode."""
import openai
client = openai.OpenAI()
text = "\n".join(
f"Action: {a['action']}\nResult: {a['result']}"
for a in self.working[:-2]
)
response = client.responses.create(
model="gpt-5.6-terra",
input=(
"Summarize these agent actions in 3-5 key points. "
"Keep important facts, results, and decisions.\n\n"
f"{text}"
),
max_output_tokens=300,
)
self.episodes.append(response.output_text)
self.working = self.working[-2:] # Keep the last 2 actions
# Limit the number of episodes
if len(self.episodes) > self.max_episodes:
self.episodes = self.episodes[-self.max_episodes:]
def build_context(self) -> str:
"""Builds the full context for the next call."""
parts = []
if self.facts:
parts.append(
"Established facts:\n" + "\n".join(f"- {f}" for f in self.facts)
)
if self.episodes:
parts.append(
"Summarized history:\n" + "\n---\n".join(self.episodes)
)
if self.working:
parts.append(
"Recent actions:\n" + "\n".join(
f"- {a['action']} → {a['result'][:500]}"
for a in self.working
)
)
return "\n\n".join(parts)Two parameters deserve discussion before you copy them as-is. The 2,000-character truncation in add_action protects against verbose tools, but it cuts blindly: if your tools return JSON, you’ll end up with an invalid object in the middle of your context. And max_episodes: 10 enforces bounded amnesia—on the eleventh summary, the first one disappears. For an agent that runs for a long time, promote important facts to facts before they fall out of the episodic window, or you’ll lose them permanently.
Taming tool results
Tools often return bulky results: web pages, SQL results, file contents. This is by far the leading cause of context saturation for an agent, and it’s also the easiest to fix because you handle it at ingestion rather than after the fact.
The function below offers three truncation strategies. The start mode suits documents where the essentials are at the top, like an article or report. The start_end mode works when the end carries the decisive information: an execution trace where the error appears last, a log file, or a paginated response where the total appears at the end. The summary mode costs an extra API call, which is justified only for long content that’s truly useful for the rest of the reasoning.
def truncate_tool_result(
result: str,
max_tokens: int = 2000,
strategy: str = "start_end",
) -> str:
"""Truncates a tool result while preserving key information."""
estimated_tokens = len(result) // 4
if estimated_tokens <= max_tokens:
return result
max_chars = max_tokens * 4
match strategy:
case "start":
return result[:max_chars] + "\n[... truncated]"
case "start_end":
half = max_chars // 2
return (
result[:half]
+ "\n[... middle truncated ...]\n"
+ result[-half:]
)
case "summary":
import openai
client = openai.OpenAI()
response = client.responses.create(
model="gpt-5.6-terra",
input=f"Summarize this data keeping the essentials:\n{result}",
max_output_tokens=max_tokens,
)
return response.output_text
return result[:max_chars]Tabular results require special handling. Cutting a SQL table in the middle removes the header or distorts the sense of volume: the agent sees twenty rows and concludes the table contains only twenty. The filter below keeps the header, a sample from the top and bottom, and—this is the key point—explicitly states the number of omitted rows and the total. The agent then knows it’s looking at an excerpt and can decide to run an aggregate query rather than reason on a sample.
def filter_sql_result(result: str, question: str) -> str:
"""Filters a SQL result to keep only what's relevant."""
lines = result.strip().split("\n")
if len(lines) <= 20:
return result
# Keep the header and first/last lines
header = lines[0]
first_lines = lines[1:6]
last_lines = lines[-5:]
return (
f"{header}\n"
+ "\n".join(first_lines)
+ f"\n[... {len(lines) - 11} lines omitted ...]\n"
+ "\n".join(last_lines)
+ f"\n\nTotal: {len(lines) - 1} rows"
)When multiple agents work together
When several sub-agents work in parallel, the temptation is to give them all the full context, just to be safe. That’s the best way to multiply your bill by the number of agents while degrading their performance, since a cluttered context makes the model less precise on its own task. Instead, share the minimal context: the global objective so the agent understands what it’s contributing to, its own role, common constraints, and a bounded excerpt of results produced by its peers.
@dataclass
class SharedContext:
"""Context shared between sub-agents."""
objective: str
constraints: list[str]
partial_results: dict[str, str] = field(default_factory=dict)
def context_for_sub_agent(self, role: str) -> str:
"""Generates a targeted context for a sub-agent."""
return (
f"Global objective: {self.objective}\n"
f"Your role: {role}\n"
f"Constraints: {', '.join(self.constraints)}\n"
f"Results from other agents:\n"
+ "\n".join(
f"- {k}: {v[:200]}"
for k, v in self.partial_results.items()
)
)The 200-character limit per partial result reflects an intention: a sub-agent needs to know what others found, not how they found it. If your agents need the full detail of neighboring results, you probably need to rethink how you’ve split the roles.
Key takeaways
- Structure agent memory in three levels: working, episodic, semantic
- Compact automatically when working memory exceeds the threshold
- Truncate tool results before injecting them into context
- Always signal that a result has been truncated, and by how much
- For multi-agent setups, share minimal context targeted by role