How Compaction Works in PiPi 中的压缩（Compaction）是如何工作的

If you have ever had a long coding session in a coding agent like Pi, Claude Code, or Codex, you will have triggered a compaction. In this post we explain how compaction works and when Pi needs to compact.如果您曾在 Pi、Claude Code 或 Codex 等编程智能体中进行过长时间的编码会话，那么您一定触发过压缩。在这篇文章中，我们将解释压缩的工作原理以及 Pi 何时需要进行压缩。

An LLM conversationLLM 对话

Large language models (LLMs) have limited context windows. The context window is what the model can "see" while producing a response. The transformer architecture used by LLMs limits how much input they can process. The input for a coding agent session includes all the previous messages and tool calls, and this keeps growing as you work. Once it exceeds the context window, the LLM rejects the request.大语言模型（LLM）的上下文窗口是有限的。上下文窗口是指模型在生成响应时能够“看到”的内容。LLM 所使用的 Transformer 架构限制了它们处理输入的能力。编程智能体对话的输入包含之前所有的消息和工具调用，并且会随着您的工作不断增长。一旦超过上下文窗口，LLM 就会拒绝请求。

When working interactively with a coding agent like Pi, the agent sends requests to an LLM and receives responses. Each request includes a system prompt, loaded files such as AGENTS.md, tool definitions, and the conversation history.当与 Pi 等编程智能体进行交互式工作时，智能体会向 LLM 发送请求并接收响应。每个请求都包含系统提示词、已加载的文件（如 AGENTS.md）、工具定义以及对话历史记录。

A coding agent's first LLM request contains this initial context, along with a first user message.编程智能体的第一个 LLM 请求包含此初始上下文以及第一条用户消息。

request 1:
[system][tools][user]

This starts a turn. The LLM may first return an assistant message containing tool calls. The agent program executes them and sends a new request to the LLM containing the complete conversation, now including the tool results. We get back another assistant message. The turn is finished when the assistant has completed generating output.这开启了一个轮次。LLM 可能会先返回一条包含工具调用的助手消息。智能体程序执行这些调用，并将包含完整对话（现在包括工具调用结果）的新请求发送给 LLM。随后我们收到另一条助手消息。当助手完成输出生成时，该轮次结束。

after request 1:
[system][tools][user][assistant: tool call][tool result][assistant]
                     <------------------->     ^        <--------->
                     returned by LLM           |        returned by LLM
                                               |
                                     produced by the agent

We continue working, and send another message.我们继续工作，并发送另一条消息。

request 2:
[system][tools][user][assistant: tool call][tool result][assistant][user]
                                                                     ^
                                                               new user message

Each turn expands the conversation. Eventually, the history exceeds the context limit. The next request then returns an error such as Request exceeds the maximum size.每一轮都会扩展对话内容。最终，历史记录会超过上下文限制。下一次请求就会返回类似“请求超过最大大小”的错误。

[system][tools][user][assistant][....][tool result][user]
                                                      ^
                                             exceeds context window

Handling context overflow处理上下文溢出

When we cannot continue with the existing conversation as-is, we have two choices.当我们无法按原样继续现有的对话时，有两个选择。

1. We can start a new, empty conversation without the accumulated context. This discards the history, including prior decisions and unresolved work. It might still be a good idea to do, because the performance of LLM outputs decrease as the context size grows.我们可以开始一个新的空对话，不再包含累积的上下文。这会丢弃历史记录，包括之前的决定和未完成的工作。但这样做通常是个好主意，因为 LLM 的输出性能会随着上下文大小的增加而下降。
2. We can create a smaller representation of the conversation context, since we want to keep this conversation going. That is what compaction does.由于我们希望保持对话的连续性，我们可以创建对话上下文的缩减表示。这就是压缩（Compaction）的作用。

Compaction压缩

In theory, there are many ways to implement compaction. For example, we can write a deterministic function which keeps some of what is in the conversation and discards the rest. In practice, though, implementations of compaction use an LLM request to summarize the conversation history.从理论上讲，实现压缩的方法有很多。例如，我们可以编写一个确定性函数，保留对话中的一部分内容并丢弃其余部分。但在实践中，压缩的实现通常会使用 LLM 请求来总结对话历史记录。

Compaction replaces part of the history with a compressed representation, leaving room for additional messages and tool calls.压缩会将部分历史记录替换为压缩后的表示形式，从而为后续的消息和工具调用留出空间。

[system][tools][compaction result][user]
                                    ^
                               new message

Pi's implementationPi 的实现

Let's look more closely at how Pi specifically implements compaction.让我们更仔细地看看 Pi 具体是如何实现压缩的。

When conversations grow too long, Pi uses compaction to summarize older content while preserving recent work. Compaction is triggered when the context limit is nearing the total size of the context window. It can also be manually triggered using the /compact command.当对话变得过长时，Pi 会使用压缩功能来总结旧内容，同时保留最近的工作。当上下文大小接近上下文窗口的总限制时，就会触发压缩。也可以使用 /compact 命令手动触发。

Pi checks for auto-compaction after a turn ends. Until then, each request extends the existing prompt and can reuse its cached prefix. Pi may also compact mid-turn, if it encounters a context overflow error.Pi 会在每一轮结束后检查是否需要自动压缩。在此之前，每个请求都会扩展现有的提示词并重用其缓存的前缀。如果遇到上下文溢出错误，Pi 也可能在轮次中间进行压缩。

When compacting, Pi retains some number of recent messages unchanged.在压缩时，Pi 会保留一定数量的近期消息而不做改动。

before compaction:
[system + tools][older turns][recent retained messages]

The number of retained messages varies because Pi uses a configurable token budget. Pi's current default of 20 thousand tokens comes out to roughly 5 to 20 turns. All the messages before this cut point are extracted and serialized, and will be summarized.保留的消息数量是可变的，因为 Pi 使用了可配置的 Token 预算。Pi 目前默认的 20,000 Token 大约相当于 5 到 20 轮对话。该截断点之前的所有消息都会被提取、序列化并进行总结。

Pi's compaction promptPi 的压缩提示词

The ideal outcome of a good summarization for a coding agent is like a handoff briefing from one shift to the next. Pi's compaction prompt focuses on the fact that there is a lot in the existing context that is no longer relevant. We should only keep around what is still important context for the next LLM request.对于编程智能体而言，理想的总结效果就像是交接班时的简报。Pi 的压缩提示词重点在于识别现有上下文中不再相关的内容。我们应该只保留对下一次 LLM 请求仍然重要的上下文。

Pi therefore sends a different request for compaction than for regular conversation.因此，Pi 在压缩时发送的请求与常规对话请求不同。

1. The system prompt used in the standalone compaction request is different. Instead of telling the LLM "you are an expert coding assistant", we tell the LLM "you are a context summarization assistant."独立压缩请求中使用的系统提示词是不同的。我们不再告诉 LLM “你是一个专业的编程助手”，而是告诉它 “你是一个上下文总结助手”。
2. The user message in the compaction request is also different. It requests "a structured summary of this conversation branch for context when returning later." The prompt specifies sections for goal, progress and key decisions.压缩请求中的用户消息也不同。它要求“为后续返回时提供上下文，对该对话分支进行结构化总结”。提示词中指定了目标、进度和关键决策等部分。
3. It's a standalone request that doesn't use any of the existing conversation history, which means it can use a different LLM model without incurring any unnecessary cost.这是一个独立的请求，不使用任何现有的对话历史记录，这意味着它可以选用不同的 LLM 模型，而不会产生任何不必要的额外成本。

The result of the compaction is appended to the Pi session as a compaction entry, and the session can now continue. After the compaction request, the context has been compressed.压缩的结果会作为一条压缩条目附加到 Pi 会话中，随后会话即可继续。在压缩请求之后，上下文已被成功压缩。

after compaction:
[system][tools][summary][recent turns][new user message]

There is now room in the conversation context for many more messages.现在对话上下文中有了更多空间来容纳后续消息。

Pi stores the compaction summary as plain text in the session. This keeps the compacted context readable and portable, since we can switch models in Pi and continue using the summary.Pi 将压缩摘要以纯文本形式存储在会话中。这使得压缩后的上下文既可读又可移植，因为我们可以在 Pi 中切换模型并继续使用该摘要。

Compaction and prompt caching压缩与提示词缓存

Prompt caching is used by LLM providers to make repeated requests in the same conversation less expensive. In an active coding session, we pay less for the context that has already been generated by the model. This caching requires an exact prefix match, so compacting a session will break the prompt cache.LLM 提供商使用提示词缓存来降低同一对话中重复请求的成本。在活跃的编码会话中，我们为模型已经生成过的上下文支付的费用更少。这种缓存需要精确的前缀匹配，因此压缩会话会破坏提示词缓存。

cached before compaction:
[system][tools][older history][recent retained turns]
<-------------------- cached prefix -------------------->

first request after compaction:
[system][tools][summary][recent retained turns][new user message]
<-- reusable -->^
                |
        first changed token
                |
                +-- everything after this point must be recomputed

The retained turns contain the same tokens, but they now follow a different prefix. Their previous cached state therefore cannot be reused.保留的轮次包含相同的 Token，但它们现在跟随的是不同的前缀。因此，它们之前缓存的状态无法被重用。

New requests after compaction will benefit from prompt caching again.压缩后的新请求将再次受益于提示词缓存。

Experiment实验

Since Pi is extensible and malleable, you can replace its compaction with your own. To test a different compaction mechanism, ask Pi to create an extension with a custom compaction prompt.由于 Pi 具有可扩展性和灵活性，您可以将其压缩机制替换为您自己的版本。要测试不同的压缩机制，请让 Pi 创建一个带有自定义压缩提示词的扩展。