Restart the Conversation, or Compact the History? How Pi Solves Automatic Context Compaction

Models Don't Remember, They Replay Context

Imagine you ask the agent to fix a payment issue, saying only, “Check why checkout returns 502.” But the model actually receives more than that: the system prompt, tool definitions, and project rule files. When it calls the log tool, the tool results also enter the history; when you add another clue, the next request carries all previous content again.

The agent isn't permanently remembering the work in some hidden brain region. It's re-presenting the current work state to the model on every request.

Each step makes sense, but the cumulative result is one: history grows monotonically, and the task might not be done yet. When the context can't hold any more, the agent can't keep carrying the past along verbatim; it must decide what content is still worth keeping.

Restart the Conversation, or Compact the History

Once the current session can't continue as-is, the agent has two basic paths. It can start a new session, immediately reclaiming all space, but goals, completed work, key decisions, and unresolved issues also leave. Or it can do compaction: replace part of the old history with a shorter representation and continue along the current task.

Pi chooses the second path. Here “compression” isn't like a zip archive that can be losslessly restored later. Pi typically has the model write a summary of old history; space comes from trade-offs: details are lost, wording is condensed, and information that appeared only once may fade over time.

So a more accurate name for compaction is lossy work-state migration. It doesn't aim to preserve every sentence, but to ensure the next model still knows: what the goal is, what's been done, which decisions can't be reversed, where things are stuck, and what to do next.

How Pi Splits Old and New Context

Pi doesn't summarize the entire conversation in one go. It first counts back from the latest message and draws a cut line. According to the current official docs and the pinned source version, automatic compaction triggers when contextTokens > contextWindow - reserveTokens. By default, it reserves 16,384 tokens for the upcoming response and tries to keep the most recent 20,000 tokens intact.

20k is a budget, not a fixed number of messages. If the recent turns are stuffed with logs and code, fewer turns can be kept. The original article roughly converts it to 5–20 turns, but that's not a guarantee.

Content before the cut is extracted, serialized, and summarized; content after the cut still enters the new request in original order. Pi typically checks for automatic compaction after a turn ends. If the model returns an over-limit error mid-turn, it might compact on the spot and retry.

Users can also manually run /compact [instructions] to compact early when needed, with additional instructions to remind the summarizer to preserve certain information. This entry point can change timing and focus, but it still can't turn a lossy summary into unlimited memory.

The Summary Is a Work Handoff

A regular summary cares about “what was just talked about.” A work handoff cares about: can the next person start working right away.

Pi initiates a separate model request, changing the role to context summarization assistant, instructing it not to continue the current conversation but to output a structured checkpoint. The current prompt requires preserving six categories:

• Goal: What the user ultimately wants to accomplish.
• Constraints & Preferences: Limits and preferences that must not be violated.
• Progress: Completed, in-progress, and blocked items.
• Key Decisions: Key choices and their rationale.
• Next Steps: The order in which to proceed.
• Critical Context: Paths, functions, errors, and other precise information.

The generated result is written back to the session as a plain-text CompactionEntry. The new request then becomes:

[system] [tools] [summary] [recent turns] [new user]

The beauty of plain text is simple: a human can check whether the summary missed any key decisions, and Pi can continue using it after switching models. What's migrated is visible work state, not some uncheckable memory inside a model.

But this handoff only preserves what the summarizer recognized. If an important constraint appeared only once very early, it may still fade through repeated compactions. Truly critical project rules are best written into AGENTS.md, task status, code, or other persistent records.

Prefix Changes, Old Cache Goes Stale

Compaction frees up space, but it also changes the most sensitive part of each request: the prefix.

Prompt caching from model providers typically relies on exact prefix matching. Before compaction, the next request mostly continues appending to the old history; after compaction, earlier history is replaced by a new summary. Even if the recent turns haven't changed a single word, what precedes them is different, so the old cache can't be reused from the same position.

This isn't a permanent loss of cache. The compacted structure becomes the new stable prefix, and subsequent requests can accumulate cache on top of it again. This also explains why Pi tries to compact after a turn ends: let this turn's continuous requests enjoy the old prefix's cache benefits, then rewrite the context.

Can We Get Straight to Work After Compaction?

Pi's trade-offs are now complete: keeping all history hits the context limit; compacting history loses details and costs a cache recompute; starting a completely new session abandons work continuity altogether.

To judge whether a compaction is good, ask four questions: How much recent work remains intact? Does the summary preserve goals, decisions, progress, and next steps? Can the result be inspected by a human and migrated between models? Does the trigger timing strike a reasonable balance between remaining space, overflow risk, and cache cost?

Good compaction isn't “remembering all the past.” It's accepting that forgetting is inevitable, then doing your best to hand off the work state that truly affects the next step.