Compaction Failed: Conversation Could Not Be Reduced Below the Context Limit

Compaction Failed: Conversation Could Not Be Reduced Below the Context Limit

Compaction failed: conversation could not be reduced below the context limit. If you build agent memory systems or RAG pipelines, you've hit that wall and stared at it. The summarizer ran, the tokens didn't shrink enough, and now your agent is stuck with a full context window and no way forward. This guide is for you, not for someone casually using a chatbot. It explains what the error actually means, why it happens, and what you can and cannot do about it. GigaRAG handles agent memory and RAG pipeline context management, but even with good tooling, some compaction failures are architectural and won't budge with configuration tweaks alone.

In This Guide

• What Does 'Compaction Failed: Conversation Could Not Be Reduced Below the Context Limit' Mean?
• Compaction vs Pruning for Context Reduction
• Why Compaction Fails: Root Causes in Agent Memory and RAG Pipelines
• Compaction Failed: Conversation Could Not Be Reduced Below The Context Limit: A Step-by-Step Guide
• How Context Compaction Works in Agent Memory Systems
• Step-by-Step Troubleshooting for Compaction Failed Errors
• Compaction Failed: What You Cannot Do or Should Not Expect
• Alternatives to Compaction: Context Editing, Pruning, and RAG-Based Memory
• Token Optimization Strategies to Prevent Compaction Failures
• Configuring Compaction in Agent Memory and RAG Pipelines
• Common Mistakes When Compaction Failed: Conversation Could Not Be Reduced Below the Context Limit
• Final Thoughts: Building Resilient Agent Memory Systems

What Does 'Compaction Failed: Conversation Could Not Be Reduced Below the Context Limit' Mean?

Compaction failed: conversation could not be reduced below the context limit means the summarization step that compresses your conversation history could not shrink it enough to fit within the model's remaining token budget. The error appears when the system tries to compact, runs the summarizer, and still ends up with more tokens than the context window can hold.

What is context compaction?

Context compaction is the process of summarizing conversation history to free up space in the context window. When your conversation approaches the token limit, the system takes the full history, runs it through a summarizer model, and replaces the original messages with a shorter summary. The goal is simple: keep the essential information while cutting the token count.

In agent memory systems, compaction happens automatically. The agent detects it's near the limit, triggers a compaction step, and continues working with the summarized history instead of the raw messages. It's a stopgap, not a solution. It buys you more turns before you hit the wall again.

What does the error message actually mean?

The error breaks into two parts. "Compaction failed" means the summarization step ran but did not produce a usable result. "Conversation could not be reduced below the context limit" means the output of that summarization, combined with whatever else is in the context window, still exceeds the token budget.

Here's what happens behind the scenes. The system has a context window of, say, 200,000 tokens. Your conversation history has grown to 195,000 tokens. The system triggers compaction, expecting the summarizer to cut that down to maybe 50,000 tokens. But the summarizer returns a summary that's still 180,000 tokens, or the summarization itself fails partway through. Either way, the result doesn't fit. The system reports the failure rather than silently truncating your history.

Why 'could not be reduced below the context limit' is a symptom, not a cause

The phrase "could not be reduced" points at the summarizer, but the summarizer is rarely the root problem. The real issue is usually upstream: the conversation history is too dense, too long, or structured in a way that resists compression. A summarizer can only compress so much before it starts losing critical information. When it refuses to go further, it's often protecting you from a worse outcome: a summary so lossy that the agent forgets what it was doing.

The honest answer is that this error is a signal. It tells you the conversation has outgrown what compaction can handle. Fixing it means addressing the underlying context management problem, not just retrying the compaction step.

[!note] Compaction reduces token count by summarizing or rewriting content, but it cannot shrink content that is already irreducible — such as a single tool output larger than the context window, or a pinned system prompt that must remain verbatim. If the irreducible portion alone exceeds the limit, compaction will always fail.

Compaction vs Pruning for Context Reduction

Why Compaction Fails: Root Causes in Agent Memory and RAG Pipelines

Compaction fails for four reasons, and they compound. Token budget exhaustion is the trigger. Dense conversation history is the fuel. Summarizer limitations are the ceiling. Unbounded context growth is the architecture underneath all of it.

Token budget exhaustion and context window limits

Every model has a hard ceiling on tokens it can hold at once. When your conversation plus system prompt plus tool outputs approach that ceiling, compaction fires. But the budget isn't just the raw window size. It's what's left after everything else takes its share.

Here's the catch: compaction itself consumes tokens. The summarizer needs the full history as input, which means you need headroom to run the summarization at all. If you're at 98% of the window when compaction triggers, the summarizer may not have enough room to process the history it's supposed to compress. The failure isn't the summarizer being bad. It's the system waiting too long to act.

Conversation history too dense or too long to summarize

Some conversations resist compression. Long code reviews with inline diffs, debugging sessions with stack traces, multi-step reasoning chains where each step depends on the last. These aren't verbose. They're dense. Every token carries load-bearing information.

A summarizer can cut filler. It can't cut meaning. When the history is 150,000 tokens of tightly coupled technical detail, no summarizer will reduce it to 20,000 tokens without dropping something the agent needs. The compaction step runs, produces a summary, and the summary is still too big. That's the error.

Summarizer model limitations and failure modes

The summarizer is usually a smaller, cheaper model than the main agent. That's intentional: you don't want to spend a full model call on housekeeping. But smaller models have smaller context windows and weaker reasoning.

Two failure modes show up. First, the summarizer's own context window is smaller than the history it's asked to compress. It can't even read the whole conversation, so it summarizes what it can see and leaves the rest untouched. Second, the summarizer produces a summary that's nearly as long as the input because it doesn't know what to drop. It errs toward keeping everything, which defeats the purpose.

Architectural issues: unbounded context growth in agent memory

The deepest problem is architectural. Many agent memory systems treat the context window as a buffer that fills up and gets compacted, filled again, compacted again. Each compaction adds a summary layer. The summaries accumulate. The raw messages get replaced, but the total token count keeps climbing because the summaries themselves never leave.

You end up with a context window full of summaries of summaries. Compaction becomes a treadmill. Each pass buys less room than the last. Eventually the system hits a state where compaction cannot reduce the total below the limit no matter how well the summarizer performs. The honest answer is that no configuration tweak fixes this. You need a different memory architecture: pruning, external retrieval, or a hard cap on retained history.

[!tip] For agent memory and RAG pipelines, set a hard token budget per tool output and per retrieved chunk at ingestion time — for example, truncate or summarize tool results before they enter the conversation history. This prevents the irreducible portion from ever growing large enough to make compaction impossible.

Compaction Failed: Conversation Could Not Be Reduced Below The Context Limit: A Step-by-Step Guide

1. Log the exact token count of the conversation before compaction and compare it to your model's context limit.
2. Identify irreducible content: pinned system prompts, large tool outputs, and oversized retrieved chunks that compaction cannot summarize away.
3. Check whether your compaction prompt itself is consuming a large share of the budget, leaving too little room for the summary.
4. Cap or truncate tool outputs and retrieved chunks at ingestion so they never exceed a fixed token budget.
5. Switch to pruning or context editing for content that cannot be meaningfully summarized, such as raw JSON or code blocks.
6. Re-run compaction with a smaller target and verify the result fits below the context limit before sending to the model.
7. Add a fallback path that prunes oldest messages if compaction still fails, so the pipeline degrades gracefully.

How Context Compaction Works in Agent Memory Systems

Compaction is a pipeline, not a single step. When it works, it buys you room. When it fails, you get the error. Understanding the mechanics tells you where to look first.

The compaction trigger: approaching the context limit

Compaction doesn't run constantly. It fires when token usage crosses a threshold, usually 80 to 90 percent of the context window. The system detects that headroom is running out and kicks off a summarization pass to reclaim space.

The trigger matters because it determines how much room the summarizer has to work with. Trigger too early and you compact history the agent still needs. Trigger too late and the summarizer can't read the full history it's supposed to compress. Most systems default to a fixed percentage, but the right threshold depends on how fast your conversations grow and how dense they are.

How summarization reduces conversation history

The summarizer reads the raw conversation and produces a condensed version. A good summary keeps decisions, constraints, and unresolved questions. It drops greetings, repeated explanations, and superseded code.

Here's what happens behind the scenes. The system splits the history into chunks the summarizer can handle. Each chunk gets summarized independently. Then the summaries get merged, sometimes in multiple passes. The output replaces the raw messages in the context window.

The catch is that summarization is lossy by design. The summarizer has to guess what matters. When it guesses wrong, the agent loses information it needs. When it guesses conservatively and keeps too much, the summary stays large and compaction fails.

Interaction between compaction and RAG retrieval

RAG pipelines add a layer. Instead of keeping everything in the context window, a RAG system stores conversation history in a vector store and retrieves relevant chunks on demand. Compaction and retrieval serve different jobs.

Compaction keeps the active context small. Retrieval brings back what the agent needs from external storage. They work best together: compaction handles the live conversation, RAG handles the long tail. But they can conflict. If compaction summarizes a conversation and the summary replaces the raw text, the vector store may still hold the old version. The agent ends up with two versions of the same history: a compressed one in context and a stale one in retrieval.

The fix is to treat compaction output as the new source of truth. When compaction runs, update the vector store with the summary, not just the context window. Otherwise your retrieval system feeds the agent information that no longer matches what it's working from.

Step-by-Step Troubleshooting for Compaction Failed Errors

When you hit "compaction failed: conversation could not be reduced below the context limit," the error is telling you one thing: the summarizer could not shrink the history enough to fit the remaining budget. The fix depends on why. Work through these steps in order. The first three are quick checks. The last two are architectural.

Step 1: Check current token usage and context window size

Before touching anything, get the numbers. Check how many tokens the conversation is currently using and what your model's context window actually is. If you're at 95 percent of a 128k window and the summarizer only reclaimed 2k tokens, the problem is obvious: you waited too long to trigger compaction.

The decision point is simple. If token usage is above 90 percent of the window, lower your compaction threshold so it fires earlier. If usage is below 80 percent and compaction still failed, the summarizer itself is the problem. Move to Step 2.

Step 2: Identify what is consuming the most tokens

Not all tokens are equal. A conversation full of short exchanges compacts differently than one with three 10k-token code blocks. Look at the raw history and find the biggest consumers.

Common culprits: pasted logs, full file contents, repeated error traces, and tool outputs the agent never used. If a single message or tool result is eating 30 percent of your budget, compaction can't fix that. You need to truncate or drop it before summarization runs. The honest answer is that compaction works on conversation structure, not on one giant blob. A 40k-token paste will survive summarization as a 30k-token summary because the summarizer can't safely discard what it can't parse.

Step 3: Test with a shorter conversation to isolate the issue

Take the same agent, same configuration, and run a conversation that's half the length. If compaction succeeds on the shorter history, the failure is load-related: your history is too dense or too long for the summarizer to handle in one pass. If compaction fails even on a short conversation, the problem is configuration or the summarizer model itself.

This test separates "too much history" from "broken compaction." It takes ten minutes and saves you from chasing the wrong fix. Don't skip it.

Step 4: Review summarizer model configuration

The summarizer is a model with its own context window and its own limits. If you're using a small model to summarize a large history, it will fail. Check three things: the summarizer's context window, the chunk size you're feeding it, and the number of merge passes.

A common failure mode is a summarizer with a 4k context window trying to summarize a 20k-token chunk. It can't read the chunk, so it produces a partial or empty summary, and compaction reports failure. The fix is to reduce chunk size or use a summarizer with a larger window. This is a configuration fix, not a band-aid, and it works.

Step 5: Evaluate architectural changes: context pruning vs. compaction

If Steps 1 through 4 don't resolve it, the problem is architectural. Your conversation history is growing faster than compaction can reclaim space. No configuration tweak fixes that.

You have two real options. Context pruning drops low-value messages before they ever reach the summarizer. It's aggressive but predictable: you decide what gets cut. Compaction tries to preserve everything in compressed form, which fails when the history is too dense. The main catch is that pruning loses information by design, while compaction loses it by accident. For agent memory systems that run long sessions, pruning plus a RAG-based memory store is usually the more reliable path. Compaction alone will keep failing as conversations grow.

Compaction Failed: What You Cannot Do or Should Not Expect

Compaction is a summarization step, not a memory manager. It has hard limits. If you treat it as a fix for every context problem, you'll keep hitting the same error.

Compaction cannot compress arbitrary history without information loss

Every summarizer has a floor. Below a certain compression ratio, the summary stops being a summary and starts being a guess. A 100k-token conversation cannot become a 2k-token summary without dropping details the agent needs. The summarizer doesn't know which details matter, so it drops them semi-randomly.

The honest answer is that compaction works best on redundant history: repeated status checks, boilerplate, tool outputs that echo the request. It fails on dense, unique information. If every message contains a distinct fact, there's nothing to compress. The information loss is the point of failure.

Compaction cannot fix architectural context growth problems

If your agent accumulates 50k tokens of history per session and your window is 128k, compaction buys you time. It doesn't change the growth rate. The next session hits the same wall.

The main catch is that compaction is reactive. It fires when you're already near the limit. A system that needs compaction every ten minutes has an architecture problem, not a summarization problem. You need pruning, external memory, or a smaller working set.

Compaction is not a replacement for context pruning or editing

Pruning drops messages you don't need. Editing rewrites them. Compaction summarizes everything, including the parts you should have dropped. That's the key difference.

A 5k-token error trace that's no longer relevant should be pruned, not summarized. Summarizing it keeps a 1k-token ghost of a useless message. Pruning removes it entirely. If you're using compaction as your only context management strategy, you're keeping information you don't need in a form you can't use.

Alternatives to Compaction: Context Editing, Pruning, and RAG-Based Memory

Compaction is one tool in a set. It's not the first tool, and it's often not the right one. The alternatives below attack the same problem from different angles: reduce what's in context, drop what doesn't matter, or move memory out of the context window entirely.

Context editing: selectively removing or rewriting conversation history

Editing means you touch the history before it hits the summarizer. You remove a message, rewrite a tool output, or replace a verbose exchange with a one-line note. The agent never sees the original.

This works when you know what's irrelevant. A failed API call that you retried successfully? Edit out the failure. A 3k-token stack trace you already fixed? Replace it with "error resolved, see commit 8f3a." You're not summarizing; you're curating.

The catch is that editing requires judgment. You need a rule or a model that knows what to keep. If you edit blindly, you lose information the agent needed later. Editing is precise but expensive in logic.

Context pruning: dropping low-value messages

Pruning is simpler than editing. You delete messages outright. No rewrite, no summary. The message is gone.

Pruning works on a scoring model: recency, relevance, message type. Tool outputs older than N turns get dropped. Status messages get dropped. Error traces get dropped. What remains is the working set the agent actually needs.

The tradeoff is permanence. A pruned message cannot be recovered unless you stored it somewhere else. If the agent needs a fact from 40 turns ago and you pruned it, the agent is stuck. Pruning pairs naturally with external memory: prune from context, store in a vector database.

RAG-based memory: externalizing context to a retrieval system

RAG-based memory moves conversation history out of the context window entirely. Messages get embedded and stored in a vector store. When the agent needs something, it queries the store and retrieves only the relevant chunks.

This is the architectural fix for unbounded context growth. Your context window stays small because it holds only the current turn plus retrieved snippets. The full history lives outside.

The cost is retrieval quality. If your embeddings are weak or your query is vague, the agent retrieves the wrong chunks and makes decisions on incomplete information. RAG also adds latency: every retrieval is a round trip. For fast, interactive agents, that delay matters.

When to use compaction vs. alternatives

Compaction is right when history is redundant and you need a lossy summary fast. It's a stopgap, not a strategy.

Editing is right when you know exactly what to remove or rewrite. Pruning is right when you have a clear rule for what's low-value. RAG-based memory is right when history is long, dense, and needs to persist across sessions.

The honest answer is that most production systems need pruning plus RAG, with compaction as a fallback for edge cases. If you're reaching for compaction first, you're probably solving the wrong problem.

Token Optimization Strategies to Prevent Compaction Failures

Compaction fails when the conversation is too big to summarize below the limit. The fix isn't a better summarizer. It's keeping the conversation from getting that big in the first place. The three strategies below reduce token pressure before compaction ever runs.

Message truncation and selective history retention

Truncation means cutting messages to a fixed length before they enter history. A tool output that's 8,000 tokens gets capped at 500. The agent sees the first 500 tokens and a marker: "output truncated." You lose the tail, but you keep the signal.

Selective retention is a step further. You don't keep every message. You keep messages that changed state: decisions, errors, user corrections. Status updates, progress bars, and repeated tool calls get dropped. A good rule is to retain the last N turns plus any message flagged as high-value.

The catch is that truncation is blunt. If the critical information sits at token 600 of an 8,000-token output, truncation destroys it. Selective retention requires a scoring rule, and scoring rules need tuning. Neither is free.

Token budgeting for agent conversations

A token budget is a hard ceiling on what the agent can hold in context at any moment. You set it below the model's context limit, not at it. If the model supports 200k tokens, budget 120k. The remaining 80k is headroom for the current turn, tool outputs, and the summarizer's own overhead.

Budgets work when they're enforced at write time. Before a message enters history, the system checks: does this push us over budget? If yes, something gets pruned or truncated first. The alternative is reactive budgeting, where you notice the overflow after it happens. Reactive budgeting is how you end up with a compaction failure.

The honest answer is that budgets only work if you track them. You need a running token count per message and per conversation. Without that, a budget is a wish.

Monitoring and alerting for context usage

Monitoring catches the problem before compaction does. Track context usage as a percentage of the window. Alert at 70%, not at 95%. At 70% you have room to prune, truncate, or trigger a RAG retrieval. At 95% you're one tool output away from failure.

Alert on rate of growth too. A conversation that jumps from 40% to 80% in three turns is a different problem than one that creeps up over fifty turns. The first is a runaway tool output. The second is normal accumulation. They need different fixes.

Keep in mind that monitoring is only useful if someone acts on the alert. An alert that fires and gets ignored is noise. Wire the alert to an automated response: trigger pruning, pause the agent, or switch to RAG retrieval. If a human has to intervene, the system already failed.

Configuring Compaction in Agent Memory and RAG Pipelines

Configuration is where compaction either works or fails silently. The defaults are rarely right for your workload. You need to set three things explicitly: which model summarizes, when compaction triggers, and how the summarized history feeds back into retrieval.

Summarizer model selection and configuration

Use a smaller, faster model for summarization than for the main agent loop. A 7B or 8B parameter model handles most conversation compression fine. The main catch is that the summarizer needs enough context to see the full history it's compressing. If your summarizer has a 32k context window and your conversation is 100k tokens, it can't summarize in one pass. You'll need chunked summarization, which produces worse results.

Set a maximum summary length. Without one, the summarizer can produce a summary nearly as long as the original. That's how you get a compaction failure on the next cycle. A good starting point is 10-15% of the original token count.

Compaction thresholds and triggers

Trigger compaction at 70-80% of your context budget, not at 95%. At 70% you have room for the summarizer's own overhead and the current turn. At 95% you're already failing.

The trigger should be a token count, not a message count. Message counts lie. Ten messages can be 500 tokens or 50,000 tokens depending on tool outputs.

Integration with RAG retrieval systems

Compaction and RAG solve different problems. Compaction shrinks what's in context. RAG moves context out entirely. They work together when the summarizer writes a condensed history to your vector store, and retrieval pulls back only what the current turn needs.

Don't let the summarizer overwrite the raw history. Store both. The summary is for fast retrieval. The raw history is for when you need to reconstruct exactly what happened. If you only keep the summary, you've traded context pressure for information loss.

Common Mistakes When Compaction Failed: Conversation Could Not Be Reduced Below the Context Limit

Most compaction failures are not configuration problems. They're workflow problems. Developers hit the error, retry the same compaction call, and watch it fail again. Here are the mistakes I see most often, and what to do instead.

Repeatedly retrying compaction without addressing root causes

Retrying the same compaction call with the same conversation history produces the same result. The summarizer already tried to compress the history and couldn't get it under the limit. Running it again doesn't change the input, so it can't change the output.

The fix is to change something before you retry. Truncate the oldest messages. Drop tool outputs you don't need. Split the history into chunks and summarize each chunk separately. If you retry without changing the input, you're just burning tokens and waiting for the same error.

Ignoring architectural context growth issues

Compaction fails when the conversation grows faster than your summarizer can compress it. That's an architectural problem, not a bug in the compaction call. If every turn adds 5,000 tokens of tool output and your summarizer can only cut 50%, you'll hit the limit no matter how well you configure the trigger.

Look at what's actually growing. Tool outputs are the usual culprit. A single file read can dump 20,000 tokens into context. If you're not pruning those before compaction runs, the summarizer is trying to compress a wall of raw data, not a conversation. Fix the growth first, then compaction has a chance.

Expecting compaction to fix unbounded context growth

Compaction is a compression step. It reduces what's already there. It doesn't stop new tokens from arriving. If your agent loops through 40 tool calls per turn, compaction will run constantly and fail constantly.

The honest answer is that some conversations should not be compacted. They should be ended, or their state should be externalized to a retrieval system. Compaction buys you headroom. It doesn't give you an infinite context window. When you find yourself compacting every turn, the problem isn't the summarizer. It's the conversation design.

Final Thoughts: Building Resilient Agent Memory Systems

Compaction failures are rarely about the summarizer. They're about what you're feeding it. When you see "compaction failed: conversation could not be reduced below the context limit," the error is telling you the conversation has outgrown the architecture, not that the compression step needs a better model.

The fixes that actually work are the boring ones. Prune tool outputs before they pile up. Set a hard cap on conversation length. Externalize state to a retrieval system instead of keeping everything in context. These changes prevent the error instead of reacting to it.

Compaction has limits. It can't compress arbitrary history without losing information. It can't fix unbounded growth. It can't replace good context design. Accept those limits and you'll stop fighting the error and start building systems that don't trigger it.

If you're building agent memory or RAG pipelines and want a tool that handles context management without the constant compaction failures, GigaRAG is worth a look. It's built for exactly this problem: keeping agent context lean while preserving what matters.

Frequently Asked Questions

What does context compaction mean?

Context compaction is the process of reducing the size of a conversation or context window by summarizing, rewriting, or otherwise shrinking its contents so it fits within a model's token limit. It is commonly used in agent memory systems to keep long-running conversations within bounds.

What does compaction mean in AI?

In AI, compaction refers to techniques that reduce the token footprint of a prompt or conversation history — typically by summarizing older turns, merging redundant messages, or abstracting detail. It is distinct from pruning, which removes content outright rather than rewriting it.

Why does compaction fail with 'conversation could not be reduced below the context limit'?

This error occurs when the irreducible portion of the conversation — such as pinned system prompts, large tool outputs, or oversized retrieved chunks — already exceeds or nearly exceeds the context limit. Compaction can only summarize content that is summarizable; it cannot shrink content that must remain verbatim or that is itself larger than the window.

What is the difference between compaction and pruning in agent memory?

Compaction rewrites and summarizes content to reduce tokens while attempting to preserve meaning. Pruning removes or truncates content entirely. Compaction is better for redundant history; pruning is better when you need a hard token ceiling and can afford to lose detail.

When should I use context editing instead of compaction?

Use context editing when you need precise control over what stays in the window — for example, keeping a specific tool output verbatim while dropping everything else. Compaction is better when the history is long and mostly redundant, and you can tolerate some loss of nuance.

How do I prevent compaction failures in a RAG pipeline?

Cap the token size of every retrieved chunk and tool output at ingestion, reserve a fixed budget for system prompts, and implement a pruning fallback for when compaction cannot meet the target. Monitor token counts at each stage so you catch growth before it hits the limit.

Can compaction ever reduce a conversation to zero tokens?

1. Compaction always leaves some content behind — at minimum the system prompt and the most recent turns needed for coherence. If those alone exceed the context limit, compaction cannot succeed and you must prune or restructure the pipeline.

About GigaRAG

GigaRAG is for agent memory and RAG pipeline builders. get this right. Whether you are working through compaction failed: conversation could not be reduced below the context limit or something adjacent, we publish what we have actually tested, including where it falls short.