Restart the Conversation, or Compact the History? How Pi Solves Automatic Context Compaction
A long task doesn't keep going because the model has a good memory. It just keeps stuffing the past into each next request; when it no longer fits, Pi rewrites its own working memory.
- Compacting context rewrites earlier work into a shorter state summary to free up context space for long tasks.
- Pi by default keeps roughly the most recent 20k tokens and reserves 16,384 tokens for the next response; these numbers are Pi-specific, not industry-wide.
- Compaction is a lossy migration: it preserves work continuity but loses details and temporarily invalidates old prompt caches.
If you've had Pi, Claude Code, or Codex continuously edit code, check logs, and run tests, you've likely seen a prompt like compacting context when the task runs long enough.
It's not compressing code or deleting files. It means the current session is nearing the limit of information the model can read at once, and the coding agent is rewriting earlier work records into a shorter version so the task can continue.
This finite space is called the context window. Each time the model answers, it can only reference what fits in the window. Long tasks keep adding new messages, code, logs, tool calls, and results, so the window fills up. compacting context appears because the agent must make room for subsequent work before hitting the capacity limit.
Earendil Engineering wrote a dedicated piece, “How Compaction Works in Pi,” dissecting Pi's approach. Pi is a coding agent that runs in the terminal. The following describes its specific implementation; other coding agents face the same general problem, but their trigger thresholds, retention amounts, and summarization methods may differ.
Models Don't Remember, They Replay Context
Imagine you ask the agent to fix a payment issue, saying only, “Check why checkout returns 502.” But the model actually receives more than that: the system prompt, tool definitions, and project rule files. When it calls the log tool, the tool results also enter the history; when you add another clue, the next request carries all previous content again.
The agent isn't permanently remembering the work in some hidden brain region. It's re-presenting the current work state to the model on every request.