We benchmarked 11 different ways to manage context in our production AI tutor.
The one we expected to perform the worst ended up winning.
Turns out it was dead simple:
Just keep the full conversation history.
It beat summarization, sliding windows, and other compaction strategies on recall, cost, and latency.
The reason was prompt caching.
On DeepSeek, across a 36-turn conversation, the model processed 1.78M tokens in total.
97% of those tokens were cached.
So even though we were sending more context on every turn, keeping the full history was still cheaper than constantly rewriting or summarizing it.
And it wasn't just cheaper. It remembered more too.
The full-history setup kept 95% memory recall, while summarization and compaction dropped to 32%.
Clearing old context also caused another problem.
When we removed old tool outputs, the agent sometimes had to retrieve the same information again.
So the "optimization" meant to reduce context ended up creating more tool calls and more work.
This doesn't mean you should never compact context.
When we ran the tutor locally with a 32k context window, keeping everything stopped being possible. At that point, compaction and retrieval became necessary.
We also found some weird results with retrieval.
In one long-context stress test, dense retrieval eventually dropped to 0% recall for facts buried deep in the context, while BM25 held up much better.
That is one reason our production tutor still uses hybrid search instead of relying on embeddings alone.
We broke down all of these experiments at AI Engineer World's Fair, including when full history wins, when compaction becomes necessary, how prompt caching changes the cost equation, and what happened when we pushed the context up to 800k tokens.
Check it out here:
piped.video/watch?v=WP3hjUXd…