Context Window Management for Long-Running Agents: 7 Fixes That Cut My Token Costs ~63%

Context window management is the difference between an agent that runs for hours and one that turns into an expensive goldfish. My research agent degraded badly until I rebuilt how it handles memory. Here are the seven fixes that saved it.

๐Ÿ“‹ Table of Contents

- Why Long-Running Agents Degrade Over Time - The 7 Fixes at a Glance - Fixes 1–3: Cut What Enters the Context - Fixes 4–7: Externalize Memory and Structure the Work - A Worked Example: My Research Agent - Pre-Deployment Checklist - FAQ

๐Ÿšจ Why Long-Running Agents Degrade Over Time

I've been running an autonomous research agent for the past six months. It monitors industry feeds, verifies claims against primary sources, and drafts briefings while I sleep. For the first few weeks it was brilliant. Then the sessions got longer, and the agent got dumber.

The pattern is called context rot, and it hits every long-running agent eventually. As the window fills with old tool outputs, stale search results, and half-relevant file dumps, the model's attention spreads across more and more tokens. Instructions from the start of the session lose their pull. In my logs, once a session passed the two hour mark, the agent was redoing work it had already finished in 40–60% of its turns: re-reading files it had summarized, re-running searches it had completed.

The cost side is just as ugly. Every turn re-sends the entire conversation, so a bloated context taxes you on every single API call. When I audited my transcripts, raw tool outputs were the biggest share of context growth by far. Most of those tokens were dead weight: search results the agent had already extracted, HTML it had already parsed, error traces from retries that later succeeded.

The fix isn't a bigger context window. Frontier models now ship with 200k and even 1M token windows, and long-running agents still rot inside them because attention quality degrades well before the window fills. The fix is treating context as a finite budget you actively manage. That mindset shift is what the seven fixes below deliver.

๐Ÿ—บ️ The 7 Fixes at a Glance

Before the details, here's the full list so you can see the shape of the work. Each fix stands alone, and you can adopt them one at a time. I've ordered them by where they act: the first three control what enters the context, and the last four restructure how the agent works around a limited window.

If you only have an afternoon, ship Fix 1 and Fix 2. They're the smallest changes with the largest payoff, and neither requires touching your agent's core logic.

# Fix What it does Effort
1 Threshold-based compaction Summarizes old turns before the window fills Low
2 Stale tool result truncation Replaces consumed tool outputs with one-line stubs Low
3 Just-in-time retrieval Loads data on demand instead of pre-stuffing it Medium
4 File-based memory Persists decisions and findings outside the window Medium
5 Sub-agent delegation Isolates deep dives in disposable clean windows Medium
6 Progress ledger Tracks task state in a structured, rewritable block Low
7 Checkpoint and restart Plans fresh sessions with a handoff summary Medium

✂️ Fixes 1–3: Cut What Enters the Context

These three fixes attack the input side. Your agent's window fills up because too much gets written into it and nothing ever leaves. Each fix below reverses one of those defaults.

Fix 1: Threshold-Based Compaction

Compaction summarizes the oldest portion of the conversation and replaces it with a compressed version. The mistake most people make is triggering it too late. If you compact at 95% capacity, you're already deep into degraded-attention territory and the summarizer itself has to work inside a rotten context. I trigger at 70–80% of the window, which leaves headroom for the summary pass to run well.

What you preserve matters more than how much you compress. My compaction prompt keeps decisions made, constraints discovered, open tasks, and unresolved errors. It drops raw tool outputs, abandoned approaches, and anything the progress ledger (Fix 6) already records. A bad summary that loses a key decision costs you more than the tokens it saved.

Fix 2: Truncate Stale Tool Results

Tool outputs are write-once, read-once in most agent loops. The agent fetches a page, extracts what it needs, and never looks at the raw HTML again. Yet that HTML rides along in the context for the rest of the session, billed on every subsequent call.

My fix: once a tool result is more than a few turns old, replace it with a one-line stub like "[fetched anthropic.com/engineering, 41k chars, key points extracted above]". The stub preserves the fact that the action happened, so the agent doesn't repeat it. In my transcripts this single change removed more tokens per session than anything else on this list.

Fix 3: Just-in-Time Retrieval

Early on, I pre-loaded everything my agent might need: config files, reference docs, prior briefings. It felt helpful. It was actually the fastest way to burn the window before any work started.

The better pattern stores lightweight references and fetches content only when a task requires it. Keep file paths instead of file contents. Keep article URLs instead of article text. Keep database IDs instead of full records. The agent pays the retrieval cost exactly once, at the moment of use, instead of carrying speculative content forever. My agent's session start context dropped from roughly 30k tokens to under 4k after this change.

๐Ÿง  Fixes 4–7: Externalize Memory and Structure the Work

Trimming input only gets you so far. At some point a long-running agent needs more state than any window can hold, and the answer is moving that state outside the model entirely.

Fix 4: File-Based Memory

Give the agent a NOTES.md file and a standing instruction: write down every durable fact, decision, and finding as you go. The file lives on disk, costs nothing while unread, and survives restarts and crashes.

The discipline that makes this work is writing before compaction, not after. If the agent records a finding the moment it confirms it, compaction can safely drop the raw evidence trail. My agent's notes file also became an unexpected debugging asset: when a briefing came out wrong, I could trace exactly which fact it recorded and when.

Fix 5: Sub-Agent Delegation

Some tasks are context bombs. Verifying one claim against primary sources can mean fetching five pages and reading thousands of lines. Doing that in the main loop poisons the window for everything that follows.

Instead, spawn a sub-agent with a clean window, a narrow brief, and an explicit output contract: "Return under 500 tokens, findings only." The sub-agent might chew through 50k tokens exploring, but those tokens die with it. Only the tight summary enters the main context. My verification pipeline runs three of these workers in parallel, and the main agent stays lean regardless of how messy the sources are.

Fix 6: A Progress Ledger

A progress ledger is a small structured block the agent rewrites every cycle: tasks done, tasks in progress, tasks blocked, and why. Unlike free-form notes, it has a fixed shape and a fixed location, so it never grows unbounded.

The ledger earns its keep right after compaction. Summaries are lossy, and the most dangerous loss is task state, since a confused agent will happily redo finished work. Because the ledger is rewritten rather than appended, it stays under a few hundred tokens no matter how long the session runs.

Fix 7: Checkpoint and Restart

Here's the counterintuitive one: don't fight for infinite sessions. A fresh context with a good handoff beats a ten-hour session that's been compacted four times. Each compaction pass compounds information loss, and by the third or fourth round the agent is working from a summary of a summary.

I restart my agent every few hours on a schedule. Before shutdown it writes a handoff summary (template in the next section), and the new session boots from that summary plus NOTES.md plus the progress ledger. Restarts went from a failure mode to a hygiene practice.

๐Ÿงช A Worked Example: My Research Agent

Numbers first, then receipts. Over the past six months, my research agent's daily token spend fell from $9.80 to $3.60, a drop of ~63%. One honest note on attribution: I measured that figure after shipping all seven fixes together, so it reflects the full set of changes rather than compaction alone. Tool result truncation and just-in-time retrieval drove the largest share in my logs, but I can't cleanly split the credit between individual fixes.

The quality gains were harder to measure but easier to feel. Repeated searches nearly disappeared from the transcripts. Briefings stopped omitting instructions I'd given at session start. The agent now runs a full overnight cycle without a single manual intervention, which it hadn't managed since its first month.

The piece that ties everything together is the handoff summary. It's the last thing the agent writes before a checkpoint restart, and it's the first thing the new session reads. Here's the exact template I use.

## Handoff Summary (write before every restart) **Objective:** One sentence. What is this agent ultimately producing? **Completed this session:** Bullet list, outcomes only, no process narration. **In progress:** Task name + exact next step, specific enough to resume cold. **Key decisions:** Anything a fresh session must not re-litigate. **Open questions:** Unresolved issues, with where to look for answers. **Files to reload:** Paths only, never contents. The new session fetches on demand.

✅ Pre-Deployment Checklist

Run through this before you let an agent loose on a multi-hour task. Every item maps to one of the seven fixes, and each one is checkable in under five minutes. If you can't tick at least the first four, expect context rot by hour two.

  • ✔Compaction triggers at 70–80% of the window, not at the ceiling
  • ✔Compaction prompt explicitly preserves decisions, constraints, and open tasks
  • ✔Tool results older than a few turns get replaced with one-line stubs
  • ✔Startup context loads references (paths, URLs, IDs) instead of full contents
  • ✔Agent writes durable facts to NOTES.md before compaction can erase them
  • ✔Context-heavy subtasks run in sub-agents with an output token contract
  • ✔Progress ledger is rewritten each cycle and stays under ~300 tokens
  • ✔Scheduled restarts write a handoff summary using a fixed template
  • ✔You track tokens per completed task, not just tokens per day

❓ Frequently Asked Questions

Can't I just use a model with a bigger context window?

A bigger window delays the problem without removing it. Attention quality degrades as token count grows, so a 1M token window still rots when you fill it with stale tool outputs. Larger windows also multiply cost, since every turn re-sends the whole conversation. Manage the budget regardless of its size.

Don't sub-agents make everything more expensive?

In my experience they usually pay for themselves. A sub-agent spends its exploration tokens once and then dies. Without one, those same tokens enter the main context and get re-billed on every subsequent call for the rest of the session. The longer your sessions run, the more delegation saves.

How do I know whether these fixes are actually working?

Track three metrics from your transcripts: tokens per completed task, daily spend, and the repeated-action rate (how often the agent redoes a search or re-reads a file it already processed). Tokens per task is the most honest one, because raw daily spend can drop simply because the agent accomplished less.

Do these techniques depend on a specific model or framework?

No. Compaction, truncation, external memory, and delegation are model-agnostic patterns. Some harnesses like Claude Code ship parts of this behavior built in, which saves implementation time, but you can implement every fix in this guide on any provider with a plain API loop.

๐Ÿ Final Thoughts

Context window management comes down to one principle: the window is a working set, and everything else belongs outside it. Compact early, stub out stale tool results, retrieve just in time, write durable state to files, delegate context bombs to sub-agents, keep a rewritable progress ledger, and restart on purpose with a clean handoff. Those seven fixes took my agent from a degrading money pit at $9.80 a day to a reliable overnight worker at $3.60, and the reliability gains mattered even more than the bill. If you're running agents in production, start with fixes 1 and 2 this week and measure tokens per task before and after. Then tell me what moved: drop your numbers in the comments, and subscribe to Agents at Work for the next deep dive, where I'll cover evaluation harnesses for long-running agents.

Last updated: September 04, 2026  ·  Keyword: context window management  ·  Agents at Work

Comments

Popular Posts