Lessons

from

building

Claude

Code:

Prompt

caching

is

everything

It is often said in engineering that "cache rules everything around me", and the same rule holds for agents.工程界常说“缓存主宰一切”，这条规则同样适用于智能体。

Long running agentic products like Claude Code are made feasible by prompt caching which allows us to reuse computation from previous roundtrips and significantly decrease latency and cost.像 Claude Code 这样长期运行的智能体产品之所以可行，得益于提示缓存，它允许我们复用之前往返的计算结果，显著降低延迟和成本。

At Claude Code, we build our entire harness around prompt caching. A high prompt cache hit rate decreases costs and helps us create more generous rate limits for our subscription plans, so we run alerts on our prompt cache hit rate and declare SEVs if they're too low.在 Claude Code 中，我们整个框架都围绕提示缓存构建。高提示缓存命中率能降低成本，并帮助我们为订阅计划创建更慷慨的速率限制，因此我们监控提示缓存命中率，并在其过低时宣布严重事件。

These are the (often unintuitive) lessons we've learned from optimizing prompt caching at scale.以下是我们从大规模优化提示缓存中学到的（通常不直观的）经验。

Lay out your prompt for caching为缓存布局提示

Prompt caching works by prefix matching—the API caches everything from the start of the request up to each cache_control breakpoint. This means the order you put things in matters enormously, you want as many of your requests to share a prefix as possible.提示缓存通过前缀匹配工作——API 会缓存从请求开始到每个 cache_control 断点的所有内容。这意味着内容的顺序至关重要，你需要尽可能让多个请求共享相同的前缀。

The best way to do this is static content first, dynamic content last. For Claude Code this looks like:最佳做法是静态内容在前，动态内容在后。对于 Claude Code，这看起来像：

1. Static system prompt & Tools (globally cached)静态系统提示和工具（全局缓存）
2. CLAUDE.md (cached within a project)CLAUDE.md（项目内缓存）
3. Session context (cached within a session)会话上下文（会话内缓存）
4. Conversation messages 对话消息

This way we maximize how many sessions share cache hits.这样我们就能最大化共享缓存命中的会话数量。

But this approach can be surprisingly fragile. We’ve broken this ordering before for a variety of reasons, including: putting an in-depth timestamp in the static system prompt, shuffling tool order definitions non-deterministically, and updating parameters of tools (e.g., what agents the Agent tool can call).但这种方法可能出奇地脆弱。我们曾因多种原因打破过这种顺序，包括：在静态系统提示中放入详细的时间戳、非确定性地打乱工具顺序定义、以及更新工具参数（例如 Agent 工具可以调用的智能体）。

Use messages for updates使用消息进行更新

There may be times when the information you put in your prompt becomes out of date, for example if you have the time or if the user changes a file. It may be tempting to update the prompt, but that would result in a cache miss and could end up being quite expensive for the user.有时，你放入提示的信息可能会过时，例如时间信息或用户更改了文件。你可能会想更新提示，但这会导致缓存未命中，最终可能对用户来说非常昂贵。

Consider if you can pass in this information via messages in the agent’s next turn instead. In Claude Code, we add a <system-reminder> tag in the next user message or tool result with the updated information for the model, which helps preserve the cache.考虑是否可以在智能体的下一轮中通过消息传递这些信息。在 Claude Code 中，我们在下一个用户消息或工具结果中添加一个 <system-reminder> 标签，其中包含更新后的信息，这有助于保持缓存。

Don't change models mid-session不要在会话中途切换模型

Prompt caches are unique to models and this can make the math of prompt caching quite unintuitive.提示缓存是模型特有的，这会使提示缓存的数学计算变得非常不直观。

For example, if you're 100k tokens into a conversation with Opus and want to ask a question that is fairly easy to answer, it would actually be more expensive to switch to Haiku than to have Opus answer, because we would need to rebuild the prompt cache for Haiku.例如，如果你与 Opus 的对话已经进行了 10 万个 token，并且想提出一个相当容易回答的问题，那么切换到 Haiku 实际上会比让 Opus 回答更昂贵，因为我们需要为 Haiku 重建提示缓存。

If you need to switch models, the best way to do it is with subagents; extending the above example, you could deploy a subagent that prompts Opus to prepare a "hand-off" message to another model on the task that it needs to get done. We do this often with the Claude Code’s Explore agents, which use Haiku.如果需要切换模型，最佳方法是使用子智能体；扩展上面的例子，你可以部署一个子智能体，提示 Opus 准备一条“交接”消息给另一个模型，说明需要完成的任务。我们在 Claude Code 的 Explore 智能体中经常这样做，它们使用 Haiku。

Never add or remove tools mid-session不要在会话中途添加或移除工具

Changing the tool set in the middle of a conversation is one of the most common ways people break prompt caching. It seems intuitive—you should only give the model tools you think it needs right now. But because tools are part of the cached prefix, adding or removing a tool invalidates the cache for the entire conversation.在对话中途更改工具集是人们破坏提示缓存最常见的方式之一。这似乎很直观——你应该只给模型当前需要的工具。但由于工具是缓存前缀的一部分，添加或移除工具会使整个对话的缓存失效。

Using Plan Mode to design around the cache使用计划模式围绕缓存进行设计

Plan Mode is a great example of designing features around caching constraints. The intuitive approach would be: when the user enters plan mode, swap out the tool set to only include read-only tools, but that would break the cache.计划模式是一个围绕缓存约束设计功能的绝佳例子。直观的方法是：当用户进入计划模式时，将工具集切换为只包含只读工具，但这会破坏缓存。

Instead, we keep all tools in the request at all times and use EnterPlanMode and ExitPlanMode as tools themselves. When the user toggles Plan Mode on, the agent gets a system message explaining that it's in Plan Mode and what the instructions are:  explore the codebase, don't edit files, and call ExitPlanMode when the plan is complete. The tool definitions never change.相反，我们始终保持所有工具在请求中，并将 EnterPlanMode 和 ExitPlanMode 本身作为工具。当用户切换计划模式时，智能体会收到一条系统消息，说明它处于计划模式以及指令：探索代码库，不要编辑文件，并在计划完成时调用 ExitPlanMode。工具定义从未改变。

This has a bonus benefit: because EnterPlanMode is a tool the model can call itself, it can autonomously enter plan mode when it detects a hard problem, without any cache break.这还有一个额外的好处：因为 EnterPlanMode 是模型可以自行调用的工具，它可以在检测到难题时自主进入计划模式，而不会破坏缓存。

Use tool search to defer instead of remove使用工具搜索来延迟加载而非移除

The same principle applies to our tool search tool. Claude Code can have dozens of MCP tools loaded, and including all of them in every request would be expensive, but removing them mid-conversation would break the cache.同样的原则也适用于我们的工具搜索工具。Claude Code 可能加载了数十个 MCP 工具，在每次请求中都包含它们会很昂贵，但在对话中途移除它们会破坏缓存。

Our solution: defer_loading. Instead of removing tools, we send lightweight stubs ( just the tool name, with defer_loading: true) that the model can "discover" via tool search when needed. The full tool schemas are only loaded when the model selects them. This keeps the cached prefix stable because the same stubs are always present in the same order.我们的解决方案：defer_loading。我们不移除工具，而是发送轻量级的存根（仅工具名称，带有 defer_loading: true），模型可以在需要时通过工具搜索“发现”它们。完整的工具模式仅在模型选择它们时加载。这保持了缓存前缀的稳定，因为相同的存根始终以相同的顺序存在。

You can also use the tool search tool through our API to simplify this.你也可以通过我们的 API 使用工具搜索工具来简化这一过程。

Compacting without breaking the cache在不破坏缓存的情况下进行压缩

Compaction is what happens when you run out of the context window. We summarize the conversation so far and continue a new session with that summary.压缩发生在上下文窗口用尽时。我们总结到目前为止的对话，并用该摘要继续一个新的会话。

Compaction interacts with prompt caching in ways that are easy to get wrong. To compact a conversation, you have to send the full conversation to the model so it can write a summary. The simplest way to do that is a separate API call with its own system prompt (something like "summarize this") and no tools attached, but that's exactly where the cost trap is. Prompt caching only applies when a request's prefix matches what's already cached, byte for byte, from the start. Your main conversation is cached under one system prompt and tool set; the summarization call uses a different system prompt and no tools, so the prefixes diverge at the very first token and none of the cache applies. You end up paying the full, uncached input rate for the entire conversation you're sending in — and the longer the conversation (i.e., the more you need compaction in the first place), the more expensive that one call becomes.压缩与提示缓存的交互很容易出错。要压缩对话，你必须将完整对话发送给模型，以便它编写摘要。最简单的方法是使用独立的 API 调用，带有自己的系统提示（如“总结这个”）且不附加工具，但这正是成本陷阱所在。提示缓存仅在请求的前缀与已缓存内容逐字节匹配时才生效。你的主对话在一个系统提示和工具集下缓存；而摘要调用使用不同的系统提示且没有工具，因此前缀在第一个 token 处就分叉，缓存完全不适用。你最终需要为发送的整个对话支付完整的未缓存输入费率——而对话越长（即你越需要压缩），这次调用就越昂贵。

The solution: cache-safe forking解决方案：缓存安全的派生

When we run compaction, we use the exact same system prompt, user context, system context, and tool definitions as the parent conversation. We prepend the parent's conversation messages, then append the compaction prompt as a new user message at the end.当我们运行压缩时，我们使用与父对话完全相同的系统提示、用户上下文、系统上下文和工具定义。我们在前面加上父对话的消息，然后在末尾附加压缩提示作为新的用户消息。

From the API's perspective, this request looks nearly identical to the parent's last request—same prefix, same tools, same history—so the cached prefix is reused. The only new tokens are the compaction prompt itself.从 API 的角度来看，这个请求看起来与父对话的最后一个请求几乎相同——相同的前缀、相同的工具、相同的历史——因此缓存前缀被重用。唯一的新 token 是压缩提示本身。

This does mean however that we need to save a "compaction buffer" so that we have enough room in the context window to include the compact message and the summary output tokens.但这确实意味着我们需要保存一个“压缩缓冲区”，以便在上下文窗口中有足够的空间来包含压缩消息和摘要输出 token。

Compaction is tricky but luckily, you don't need to learn these lessons yourself—based on our learnings from Claude Code we built compaction directly into the API, so you can apply these patterns in your own applications.压缩很棘手，但幸运的是，你不需要自己学习这些经验——基于我们从 Claude Code 中学到的经验，我们直接将压缩功能构建到了 API 中，因此你可以在自己的应用程序中应用这些模式。

Lessons learned经验教训

Here are a few patterns we’ve found useful for optimizing prompt caching when building an agent: 以下是我们发现对构建智能体时优化提示缓存有用的一些模式：

1. Prompt caching is a prefix match. Any change anywhere in the prefix invalidates everything after it. Design your entire system around this constraint. Get the ordering right and most of the caching works for free.提示缓存是前缀匹配。前缀中的任何更改都会使之后的所有内容失效。围绕这个约束设计你的整个系统。正确的顺序能让大部分缓存自动生效。
2. Use messages instead of system prompt changes. You may be tempted to edit the system prompt to do things like entering plan mode, changing the date, etc. but it would actually be better to insert these into messages during the conversation.使用消息而不是更改系统提示。你可能会想编辑系统提示来执行诸如进入计划模式、更改日期等操作，但实际上更好的做法是在对话过程中将这些信息插入到消息中。
3. Don't change tools or models mid-conversation. Use tools to model state transitions (like plan mode) rather than changing the tool set. Defer tool loading instead of removing tools.不要在对话中途更改工具或模型。使用工具来建模状态转换（如计划模式），而不是更改工具集。延迟加载工具而不是移除工具。
4. Monitor your cache hit rate like you monitor uptime. We alert on cache breaks and treat them as incidents. A few percentage points of cache miss rate can dramatically affect cost and latency.像监控正常运行时间一样监控缓存命中率。我们对缓存中断发出警报，并将其视为事件。几个百分点的缓存未命中率会显著影响成本和延迟。
5. Fork operations need to share the parent's prefix. If you need to run a side computation (compaction, summarization, skill execution), use identical cache-safe parameters so you get cache hits on the parent's prefix.派生操作需要共享父前缀。如果你需要运行侧边计算（压缩、摘要、技能执行），使用相同的缓存安全参数，以便在父前缀上获得缓存命中。

Claude Code is built around prompt caching from day one; for the best results when building an agent, we suggest you do, too.  Claude Code 从第一天起就围绕提示缓存构建；为了在构建智能体时获得最佳效果，我们建议你也这样做。

Get started with Claude Code today. 立即开始使用 Claude Code。

This article was written by Thariq Shihipar, a member of technical staff on the Claude Code team. 本文由 Thariq Shihipar 撰写，他是 Claude Code 团队的技术人员。

‍