Context Compaction: What to Keep When the Window Fills Up

Every agent conversation that runs long enough hits the same wall. The context window has a ceiling, the transcript keeps growing, and at some point something has to give — either you stop the agent from remembering everything, or you stop the conversation. Since stopping the conversation isn't usually an option, the real question every long-running agent has to answer, explicitly or by accident, is: when the window fills up, what gets to survive?

Most teams answer this by accident. They bolt on a generic summarization step — condense everything older than N turns into a paragraph — and move on, treating compaction as a plumbing problem rather than a design decision. It isn't. What you compress and what you preserve verbatim is one of the highest-leverage choices in the entire context-engineering stack, because get it wrong and the agent doesn't just forget things — it forgets the specific things that turn out to matter three turns later, in ways that are hard to predict from the compaction step itself.

Why naive summarization fails predictably

The default approach — summarize old turns into a shorter paragraph, keep recent turns verbatim — treats all information from a given time window as equally compressible. It isn't. A summarization pass is very good at capturing the gist of a conversation and very bad at preserving the kind of detail that only matters conditionally: an exact number, a specific constraint the user stated once, a decision that was made and never repeated because it didn't need to be.

This produces a specific and recognizable failure pattern in long agent sessions: the agent behaves correctly for the first stretch of a conversation, then starts contradicting an earlier decision or dropping a constraint right around the point where compaction kicked in. It isn't disobeying the user. It's operating on a summary that smoothed away the one clause that mattered, because a generic summarizer has no way of knowing in advance which clause that would be.

What's actually safe to compress

Some categories of context compress well, because their value is in the gist, not the specifics:

• Resolved sub-tasks. If the agent spent six turns debugging why an API call was failing and eventually fixed it, the future-relevant fact is "the API call was failing because of X, and it's now fixed" — not the six turns of diagnostic back-and-forth that got there. Compressing this into one sentence loses almost nothing.
• Exploratory dead ends. Paths the agent or user considered and explicitly abandoned are safe to summarize to "considered X, decided against it because Y" — the reasoning trace usually isn't needed again once the decision is made.
• Narrative connective tissue. The conversational scaffolding around a decision — "okay let's try that," "sounds good," "yeah go ahead" — carries no information that needs to survive compaction at all. It's the easiest category to drop and the one naive approaches usually keep by default, simply because it's part of the transcript.

What has to survive verbatim

Other categories don't compress safely, because their value is the specific wording, not the gist:

• Explicit constraints and preferences. "Don't use emoji in this document," "always use British spelling," "never touch the production branch directly" — these are exactly the kind of instruction that a paragraph summary tends to either drop or soften into something vaguer ("the user has some formatting preferences"), which is functionally the same as forgetting it.
• Numbers, identifiers, and exact values. A budget figure, an account ID, a specific version number — summarization is lossy by design, and lossy compression of a precise value is just wrong information with high confidence attached.
• Decisions with downstream consequences. If a decision three turns ago constrains what's valid four turns from now — an architecture choice, a scope boundary, a name that's now locked in — that decision needs to remain retrievable in its original form, not paraphrased.
• Anything the user stated once and clearly expects to be permanent. Users rarely repeat instructions they've already given. The absence of repetition is not a signal that something stopped mattering — it's usually a signal the user assumes it's already been retained.

The pattern across all four: what needs to survive compaction is disproportionately the low-frequency, high-consequence information — stated once, rarely repeated, and expensive to get wrong. That's precisely the information a frequency-blind summarizer is worst at preserving, because summarization naturally weights toward what's discussed at length, not what's stated once and assumed.

Structured extraction beats narrative summarization

The fix that actually works in practice isn't a better summarization prompt — it's changing what gets extracted in the first place. Instead of compressing a stretch of conversation into a shorter paragraph of prose, extract it into a small set of structured facts: constraints, decisions, open questions, key values. A structured record — "constraint: no emoji in output," "decision: using PostgreSQL not MongoDB, decided turn 12," "budget: $40,000" — survives compaction because it isn't being re-summarized every time the window compacts again; it's being carried forward as data, appended to, and only occasionally pruned when a fact is explicitly superseded.

This is a different operation from narrative compression, and it's the connective tissue between compaction and the memory-architecture question the rest of this series returns to: the right way to handle a filling context window usually isn't "summarize the prose better" — it's "stop storing prose as the thing that needs to survive, and start storing structured state." Prose is for the parts of context that are genuinely disposable. The parts that aren't disposable shouldn't be going through a lossy prose pipeline at all.

When compaction should trigger, not just what it should do

The other half of the design decision, separate from what to keep, is when compaction runs at all. A fixed turn-count trigger — "compact everything older than twenty turns" — is simple to implement and wrong in a specific, predictable way: it treats a twenty-turn conversation that covered five unrelated topics the same as a twenty-turn conversation that was one continuous thread, even though the first has far more safely-compressible material per turn than the second. A better trigger watches for actual topic or task boundaries — the point where a sub-task visibly resolves, or the user pivots to something new — and compacts at that seam, where the historical content is naturally at its most summarizable, rather than at an arbitrary token count that might land in the middle of an unresolved thread.

This matters because compaction quality and compaction timing aren't independent. Even a well-designed extraction step, run at the wrong moment, will misjudge what's safe to compress — cutting into a decision that hasn't been fully stated yet, or summarizing a sub-task that's still open. Getting the trigger right removes a lot of the pressure from getting the extraction perfect, because there's a natural point in most conversations where very little that matters is still in flight.

Compaction as a designed event, not a fallback

The teams whose long-running agents hold up over many-turn sessions treat compaction as a designed step with its own logic, tested against realistic long conversations — not a generic library call inserted once the token limit started causing errors. That means explicitly categorizing what kind of information a given stretch of conversation contains before deciding how to compress it, rather than applying one compression strategy uniformly to everything older than a fixed turn count.

It's more upfront work than calling a summarization function and moving on. It's also the difference between an agent that degrades gracefully as a conversation grows and one that works perfectly for the first thirty turns and then starts quietly contradicting itself — not because anything broke, but because something that mattered got smoothed away by a summarizer that had no way of knowing it should have been kept.

---

Part of the "Context Engineering" series on aiskill.market.