Once Claude can measure something, it can make it faster. So we kept finding more things to measure.
This August, we made the core user experience of claude.ai and the Claude desktop app about 3x faster in a two-week sprint. Users had been telling us it was slow, and they were right. We ran everything from a single Slack channel, with Claude in every thread.今年八月,我们在两周的冲刺中,将 claude.ai 和 Claude 桌面应用的核心用户体验速度提升了约 3 倍。用户一直反馈说它很慢,他们是对的。我们通过一个单一的 Slack 频道开展了所有工作,Claude 参与了每一个讨论串。
We focused on four journeys that make up 95% of user activity. At the 75th percentile, time to a typeable page on a fresh load of claude.ai went from 3.1 seconds to 0.55, starting a new Claude Code session went from 0.8 seconds to 0.3, and loading a Claude Cowork cloud session went from 2.6 seconds to 0.73. In aggregate, we estimate that saves tens of thousands of user-hours of waiting every day.我们聚焦于占用户活动 95% 的四个核心流程。在第 75 百分位下,claude.ai 首次加载至可输入页面的时间从 3.1 秒缩短至 0.55 秒,启动新的 Claude Code 会话的时间从 0.8 秒缩短至 0.3 秒,加载 Claude Cowork 云会话的时间从 2.6 秒缩短至 0.73 秒。综合来看,我们估计这每天为用户节省了数万小时的等待时间。
We used Claude Tag (beta), running an internal research model roughly comparable to Opus 5.5. Claude found bottlenecks, built benchmarks, shipped improvements, and watched every deploy. We steered by setting goals, making tradeoffs, and approving every change. With that approach, we merged more than three thousand changes without a single customer-facing incident or rollback. This post covers what we shipped, how we measured it, and the loop we built with Claude to do it safely.我们使用了 Claude Tag(测试版),运行了一个大致相当于 Opus 5.5 的内部研究模型。Claude 负责发现瓶颈、构建基准测试、发布改进方案并监控每一次部署。我们则通过设定目标、权衡取舍以及审批每一项变更来进行把控。采用这种方法,我们合并了超过三千次变更,且没有发生任何面向客户的事故或回滚。本文将介绍我们发布的内容、衡量方式,以及我们与 Claude 建立的安全协作循环。
Before the sprint, we created a Slack channel with the following standing instructions:在冲刺开始前,我们创建了一个 Slack 频道,并发布了以下常设指令:
@Claude Your job is to facilitate all things related to the performance of the claude.ai website and desktop app. Your responsibilities include monitoring deploys for performance regressions, assessing the accuracy and comprehensiveness of existing telemetry, maintaining well-curated observability dashboards, proactively implementing solutions for observed issues and low-hanging fruit, proposing performance project opportunities, and communicating with your human teammates. […]@Claude 你的工作是推动与 claude.ai 网站和桌面应用性能相关的一切事务。你的职责包括:监控部署以防性能回归、评估现有遥测数据的准确性和全面性、维护精良的观测仪表板、主动实施针对已发现问题和易于解决问题的方案、提出性能改进项目机会,并与你的人类队友沟通。[…]
The ultimate goal for this channel is for you to become as autonomous as possible, but today we know that isn’t yet possible.这个频道的最终目标是让你尽可能实现自主,但我们知道目前还无法完全做到。
We asked Claude to analyze usage data through the Datadog MCP server. It identified the four highest-impact user journeys: launching the app, starting a conversation, loading an existing conversation, and sending a message. Between web and desktop, and across our products, those journeys came to thirteen distinct measurements. To establish baselines, we added instrumentation until they were directly comparable: each started with a user interaction, ended once the result was rendered, and disambiguated client and server work.我们要求 Claude 通过 Datadog MCP 服务器分析使用数据。它确定了四个影响力最大的用户流程:启动应用、发起对话、加载现有对话以及发送消息。在网页端和桌面端以及我们的各产品之间,这些流程总计形成了十三个不同的衡量指标。为了建立基准,我们进行了插桩,直到它们可以直接对比:每个流程都以用户交互开始,在结果渲染完成后结束,并区分了客户端和服务器端的工作。
We kicked off the sprint with a list of about twenty hand-picked projects, each targeting a specific journey. Claude estimated the impact of each project in milliseconds, and we aggregated those estimates to set our targets for the sprint. Some of the projects were fairly large, but we thought we could probably achieve most of them within two weeks.我们以一份包含约二十个精选项目的列表开启了冲刺,每个项目都针对一个特定的流程。Claude 以毫秒为单位估算了每个项目的影响,我们将这些估算值汇总,设定了冲刺目标。其中一些项目规模较大,但我们认为在两周内完成大部分是可行的。
We hit twelve of the thirteen targets by day three.到第三天,我们已经完成了十三个目标中的十二个。
The planned projects landed early. For faster launches, we baked a static composer into the HTML so users can type during React initialization, and precompiled a V8 code cache so the desktop shell’s main process doesn’t recompile from scratch. For faster navigations, we kept the composer mounted between conversations, prefetched sessions when the user hovered over them, and cut sidebar re-renders by 90%.计划内的项目提前落地。为了加快启动速度,我们将静态编辑器(static composer)嵌入到 HTML 中,使用户可以在 React 初始化期间进行输入;并预编译了 V8 代码缓存,防止桌面外壳的主进程从零开始重新编译。为了加快导航速度,我们在对话之间保持编辑器挂载状态,在用户悬停时预取会话,并将侧边栏的重新渲染次数减少了 90%。
We had also left room for Claude to identify opportunities and propose new workstreams. Those workstreams quickly ramped into full projects of their own, which far exceeded our initial targets. So we set new targets, then looked for more things to measure:我们还预留了空间,让 Claude 去发现机会并提出新的工作流。这些工作流很快演变成了完整的项目,远远超出了我们最初的目标。因此,我们设定了新的目标,并寻找更多可以衡量的内容:
@Claude we’ve ended up funding nearly every project in the original projects list and more. let’s do a refresh […] what have we not explored, what can we hill climb on, where is the most opportunity at this point? […] i am open to WACKY ideas@Claude 我们最终资助了原始项目列表中的几乎每一个项目,甚至更多。让我们刷新一下 […] 还有什么是我们没探索过的,我们可以在哪些方面进行爬山算法(hill climb)优化,目前最大的机会点在哪里?[…] 我对疯狂的想法持开放态度
From the start, we knew we wanted to iterate faster than our deploy cadence. Claude could work asynchronously for many hours, even overnight, and we wanted to let it validate its prototypes without waiting for field reads. To achieve that, we looked for other ways to measure performance in the lab.从一开始,我们就知道我们希望迭代速度快于部署节奏。Claude 可以异步工作很多小时,甚至通宵达旦,我们希望让它在无需等待现场读取的情况下验证原型。为了实现这一点,我们在实验室中寻找其他衡量性能的方法。
Sam found the first lead:Sam 发现了第一个线索:
What can we do instead of wall-clock timing? Can we measure JS instruction counts for instance?除了挂钟计时(wall-clock timing),我们还能做什么?例如,我们能测量 JS 指令计数吗?
Yes. For pure-JS hot paths, literal instruction counts: run the benchmark under Valgrind with node --predictable and compare to a checked-in baseline — one run, no statistics needed.可以。对于纯 JS 热路径,可以直接进行指令计数:在 Valgrind 下运行基准测试,使用 node --predictable 参数,并与已归档的基准进行对比——只需运行一次,无需统计。
For browser paths there’s no instruction counting under Chromium, but there’s a ladder of other deterministic counts: React commits per interaction, function call counts from V8’s precise coverage, layout and style-recalc counts, DOM mutations. Which do you want first?对于浏览器路径,Chromium 下没有指令计数,但有一系列其他确定性计数:每次交互的 React 提交次数、来自 V8 精确覆盖率的函数调用次数、布局和样式重算次数、DOM 突变次数。你想先了解哪一个?
Let’s explore valgrind + Ir + --predictable in one thread and each of the browser/react benches in new threads. Ping me in all of them. You know what we want. Let’s go让我们在一个线程中探索 valgrind + Ir + --predictable,并在新线程中分别进行浏览器/react 基准测试。在所有这些测试中随时 ping 我。你知道我们想要什么。开始吧
Eleven minutes later, five threads were running, each focused on a different measurement: instruction counts, V8 call counts, React commits, style recalculations, and DOM mutations.十一分钟后,五个线程正在运行,每个线程专注于不同的指标:指令计数、V8 调用计数、React 提交、样式重算和 DOM 突变。
We treated every new benchmark with some skepticism. Each one had two jobs: first, a metric Claude could move in the lab; second, a guardrail in CI with a number that could only ratchet down. If a benchmark was flaky, or if it didn’t actually correlate with user latency, we threw it out rather than let Claude climb the wrong hill.我们对每一个新的基准测试都持怀疑态度。每一个基准测试都有两项任务:首先,作为一个 Claude 可以在实验室中优化的指标;其次,作为 CI 中的护栏,其数值只能下降。如果一个基准测试不稳定,或者它实际上与用户延迟不相关,我们会将其抛弃,而不是让 Claude 爬错山。
@Claude please prove that hill climbing against each of these can result in measurable wall clock perf wins. we’ll unship the benches for any candidates that cannot prove that@Claude 请证明针对这些指标中的每一项进行爬山优化都能带来可衡量的挂钟时间性能提升。对于任何无法证明这一点的候选指标,我们将移除其基准测试
Wall-clock time is what users feel, but it’s noisy, and milliseconds are too flaky to use as a CI gate. Instruction counts were appealing because they were deterministic, but we still needed Claude to prove they tracked wall-clock time.挂钟时间是用户的直观感受,但它充满了噪声,且毫秒级的波动太大,无法作为 CI 的门控。指令计数因其确定性而具有吸引力,但我们仍然需要 Claude 证明它们能追踪挂钟时间。
So we asked Claude to drive the count down on two hot paths: the routine that assembles a conversation’s message tree, and a scanner for status lines in Claude Code output. Claude profiled both with Valgrind and found that a quarter of the first path’s instructions were megamorphic dictionary lookups, resolving the same message ID three separate times.因此,我们要求 Claude 在两条热路径上降低计数:组装对话消息树的例程,以及 Claude Code 输出中状态行的扫描器。Claude 用 Valgrind 对两者进行了分析,发现第一条路径中四分之一的指令是巨型态(megamorphic)字典查找,即对同一个消息 ID 解析了三次。
An hour later it had cut instructions on both paths by 48% and 31%, and wall-clock time had dropped 78% and 44%. We checked in two new ratchets. From then on, any PR that raised the instruction counts of those paths failed CI, and a daily job lowered each ceiling whenever the count went down.一小时后,它将两条路径上的指令分别减少了 48% 和 31%,挂钟时间分别下降了 78% 和 44%。我们归档了两个新的限制指标。从那时起,任何增加这些路径指令计数的 PR 都会导致 CI 失败,且每天都有一个任务在计数下降时降低上限。
That led us to the central lesson of the sprint. With Claude, measuring something makes it tractable.这引出了我们本次冲刺的核心经验:有了 Claude,衡量某件事就意味着它变得可控。
Measurement used to be step zero: you’d add a metric, wait for data to roll in, and only then start to understand the problem. With Claude, it’s step one of the climb. As soon as Claude had a number to beat, it could start optimizing. This meant the highest-leverage thing we could do was find more things to measure.衡量曾经是第零步:你得先添加一个指标,等待数据汇总,然后才能开始理解问题。而有了 Claude,这就是爬山的第一步。一旦 Claude 有了一个要超越的数字,它就开始优化。这意味着我们能做的杠杆率最高的事情,就是寻找更多可以衡量的内容。
All of this ran in the same Slack channel, with multiple engineers and Claude jamming in every thread. From there, the sprint settled into a loop:所有这些工作都在同一个 Slack 频道中进行,多名工程师与 Claude 在每一个讨论串中协作。此后,冲刺进入了一个循环:
An example: someone shared a screen recording that showed sidebar rows popping in after the page loaded. Chat and Cowork rows resolved at different times, making the page feel janky. None of our existing monitors detected it. The closest we had was Cumulative Layout Shift, but each shift only scored about 0.008 — well within the good threshold of 0.1.举个例子:有人分享了一段屏幕录像,显示侧边栏行在页面加载后才弹出。聊天和 Cowork 行在不同时间解析完成,导致页面感觉卡顿。我们现有的监控工具都没有检测到这一点。我们最接近的指标是累积布局偏移(Cumulative Layout Shift),但每次偏移的得分仅为 0.008 左右——远低于 0.1 的优秀阈值。
Issac had the idea to reference the underlying Layout Instability API directly. Claude created a telemetry event that mapped the sources of each layout-shift entry to a named region (e.g. sidebar, transcript) and phase (e.g. before first paint, after typeable). It added an integration test that opened the page with a populated sidebar, held the sidebar’s data until after first paint, and failed on any shift in any named region. Claude used that as a benchmark to prove a fix: the test went red 20 of 20 runs on main, and green 20 of 20 on the PR.Issac 提出直接引用底层的布局不稳定性 API。Claude 创建了一个遥测事件,将每个布局偏移条目的来源映射到命名区域(例如侧边栏、转录文本)和阶段(例如首次绘制前、可输入后)。它添加了一个集成测试,在侧边栏已填充的情况下打开页面,将侧边栏数据保留到首次绘制后,并对任何命名区域中的偏移进行报错。Claude 将其作为基准测试来证明修复方案:该测试在主分支上 20 次运行中有 20 次报错,而在 PR 上 20 次全部通过。
After the event deployed, Claude read the field data and found that 31% of web page loads moved something after the page was usable, without any user interaction. From there, Claude worked through the causes by name: a header row that arrived late, a caret that slid sideways once the user’s name loaded, a list that moved when the scrollbar popped in. Claude fixed the top offenders as a batch, and when they were gone, it found the next batch.事件部署后,Claude 读取了现场数据,发现 31% 的网页加载在页面可用后仍有内容发生位移,且没有任何用户交互。此后,Claude 按名称排查原因:一个加载较晚的标题行、一个在用户姓名加载后向侧滑动的插入符、一个在滚动条弹出时移动的列表。Claude 批量修复了最严重的问题,修复完成后,它又找到了下一批。

That was one thread. During the sprint, we ran more than a hundred and fifty at a time.这只是一个讨论串。在冲刺期间,我们同时运行了一百五十多个讨论串。
Once the loop worked on one thread, running it on more was just a matter of opening them. Instead of closing a thread once its original request had been fulfilled, Claude would keep going. An individual thread would put up fifty, sometimes a hundred, optimization PRs. Increasingly, it was Claude, not one of us, opening new threads to chase opportunities it had found on its own, as part of a separate investigation or nightly job. Shelley, one of the engineers in the channel, observed, “[This model] is a numbers demon.”一旦循环在一个线程中生效,在更多线程中运行就只是开启它们的问题。Claude 不会在原始请求满足后关闭讨论串,而是继续进行。一个单独的讨论串可能会提交五十甚至一百个优化 PR。越来越多的时候,是 Claude 而不是我们中的某个人在开启新的讨论串,去追逐它自己发现的机会,作为独立调查或夜间任务的一部分。频道中的工程师之一 Shelley 评论道:“(这个模型)简直是个数字恶魔。”
Every measurement found something to improve. Claude ran a React hook census and found 6,900 hooks and 900 store subscriptions in the composer’s typing path, re-rendering on every keystroke. Claude counted style recalculations and found a single :root:has() selector adding 24 milliseconds to every DOM change. Claude traced code paths after first paint and found a leftover location.reload() causing half a million hidden reloads a day that none of our load metrics could see. Claude read profiler samples from idle tabs and found identical cache snapshots being cloned into IndexedDB twice a minute, all on the main thread.每一项衡量指标都发现了可以改进的地方。Claude 对 React 钩子进行了清查,发现在编辑器的输入路径中有 6,900 个钩子和 900 个存储订阅,每次按键都会触发重新渲染。Claude 计算了样式重算次数,发现一个 :root:has() 选择器为每次 DOM 变更增加了 24 毫秒。Claude 追踪了首次绘制后的代码路径,发现了一个遗留的 location.reload(),导致每天发生五十万次隐藏的重载,而我们的任何负载指标都无法察觉。Claude 读取了空闲标签页的分析样本,发现相同的缓存快照每分钟被克隆到 IndexedDB 两次,且全在主线程上。
We rarely knew where a thread would lead. In a sweep for CPU hitches, Claude noticed that highlighting a finished code block could freeze the page for about a second. It dug in the lab and found the culprit: em dashes. If a reply’s markdown contained any non-Latin-1 character, like an em dash or a curly quote, V8 stored the entire string as UTF-16, which put every syntax-highlighting regex on its slower two-byte path. Claude fixed it with a twenty-line change to copy each code block into a one-byte string before highlighting it.我们很少知道一个讨论串会引向何处。在一次 CPU 阻塞排查中,Claude 注意到高亮显示已完成的代码块可能会导致页面冻结约一秒钟。它深入实验室研究,找到了罪魁祸首:长破折号(em dashes)。如果回复的 Markdown 包含任何非 Latin-1 字符(如长破折号或弯引号),V8 会将整个字符串存储为 UTF-16,这使得每个语法高亮正则表达式都走上了较慢的双字节路径。Claude 用一个二十行的变更修复了它:在进行高亮显示前,将每个代码块复制到一个单字节字符串中。
By the second week, we could barely summarize our output into daily updates. On the busiest days, more than two hundred changes landed. Claude kept proposing new benchmarks; about a third of PRs included additional telemetry or guardrails, and each new instrument generated more threads with more opportunities.到了第二周,我们几乎无法将产出总结成每日更新。在最忙碌的日子里,有超过两百次变更落地。Claude 不断提出新的基准测试;大约三分之一的 PR 包含了额外的遥测或护栏,每一个新的工具又产生了更多带有更多机会的讨论串。
Working in one channel meant everything happened in the open. We jumped in and out of each other’s threads to debate decisions and celebrate wins. Word spread: other teams started bringing their changes into the channel to have them reviewed for performance. New projects were written in subtly more performant ways because of all the guardrails and Claude skills that had been introduced.在一个频道中工作意味着一切都在公开透明中进行。我们跳进跳出彼此的讨论串,辩论决策并庆祝胜利。消息传开了:其他团队开始将他们的变更带入该频道,以进行性能审查。由于引入了所有的护栏和 Claude 的技能,新项目在编写时就自带了更高的性能。
We’d prepared for the pace. Because nearly everything we touched was a hot path (the first paint, the composer, the transcript), we’d established our safety mechanisms up front. Every PR went through automated review with at least one human approval, unit tests always came before optimizations, and anything that could cause a user-visible problem shipped behind a short-lived feature flag.我们为这种节奏做好了准备。由于我们触及的几乎都是热路径(首次绘制、编辑器、转录文本),我们预先建立了安全机制。每个 PR 都经过自动化评审和至少一名人类审批,单元测试始终置于优化之前,任何可能导致用户可见问题的变更都放在短期的功能标志之后发布。
When the flags started piling up, we opened a thread to coordinate their rollouts and cleanup. Claude classified every flag as a kill switch or ramp, and retired each one as soon as it was safe. Across the two weeks, we introduced nearly two hundred flags, more than half of which were already cleaned up by the end.当标志开始堆积时,我们开了一个讨论串来协调它们的发布和清理。Claude 将每个标志分类为“终止开关”或“灰度发布”,并在安全的情况下立即停用每一个。在两周内,我们引入了近两百个标志,其中超过一半在结束时已经清理完毕。
We also knew that performance wins decay in a fast-moving codebase, and code ships fast at Anthropic. Once a project proved a win, we invested in ways to protect it. The static composer, for example, is brittle by design. We show users an HTML copy of the page almost immediately, and let React paint directly on top of it.我们也知道,在快速发展的代码库中,性能优势会衰减,而 Anthropic 的代码发布速度极快。一旦一个项目证明了其价值,我们就投资于保护它的方法。例如,静态编辑器在设计上就很脆弱。我们几乎立即向用户展示页面的 HTML 副本,并让 React 直接在其上方进行绘制。

If the React render is off by even a pixel, the magic is lost. So Claude built dozens of guardrails:如果 React 渲染哪怕偏移了一个像素,这种魔法就会失效。因此,Claude 构建了数十个护栏:
Not everything could be caught in the lab, so we also made use of the oldest guardrail in the book: incremental rollouts. High-risk changes were rolled out to employees first, then one percent of users, then everyone. Four hours after we released the static composer internally, a teammate shared a screen recording of a layout shift that none of our metrics could see. When he opened claude.ai in a new tab, the composer would drop — but it wasn’t our code.并非所有问题都能在实验室中捕获,因此我们还使用了最古老的护栏:增量发布。高风险变更首先向员工发布,然后是 1% 的用户,最后是所有人。在我们内部发布静态编辑器四小时后,一位队友分享了一段屏幕录像,显示了一个我们指标无法察觉的布局偏移。当他在新标签页中打开 claude.ai 时,编辑器会下移——但那不是我们的代码。
Occasionally, I’m seeing a small-ish (maybe 15–20px) vertical layout shift (pushing the composer box down) when opening claude.ai in a new tab (not so much when reloading the page). I can’t quite pin down what exactly causes that, but it’s there偶尔,当我在新标签页中打开 claude.ai 时(重新加载页面时没那么明显),我会看到一个小幅(大约 15-20px)的垂直布局偏移(将编辑器框向下推)。我不太确定到底是什么原因导致的,但它确实存在。
Found it in your recording — it’s Chrome resizing the page, not the handoff from the static composer to the real one (that measured 0 px on all 49 of your loads today).在你的录像中找到了原因——是 Chrome 在调整页面大小,而不是从静态编辑器到真实编辑器的交接(在你今天所有的 49 次加载中,该指标均为 0 像素)。
Somehow, Claude traced it to an edge case in Chrome’s speculative loading. While a user was typing the URL into the address bar, Chrome would prerender the page in the background, at the height of the current tab. On browsers managed by an organization, the new-tab page is slightly shorter because of a footer. When the user pressed Enter, the first frame of claude.ai showed that slightly shorter layout, and Chrome resized it about a tenth of a second later. Claude pinned the layout across the resize, and we added a test to simulate the prerender flow.不知何故,Claude 将其追溯到 Chrome 推测性加载中的一个边缘情况。当用户在地址栏输入 URL 时,Chrome 会在后台以当前标签页的高度预渲染页面。在由组织管理的浏览器上,由于页脚的存在,新标签页稍微短一些。当用户按下回车键时,claude.ai 的第一帧显示了那个稍微短一点的布局,而 Chrome 在约十分之一秒后调整了它。Claude 将布局固定在调整大小的过程中,我们添加了一个测试来模拟预渲染流程。
The loop was productive, but it wasn’t autonomous. Keeping it fast, safe, and on track was our job, and it had three parts.循环是高效的,但它不是自主的。保持其快速、安全且步入正轨是我们的工作,这包含三个部分。
Ambition. By default, Claude is careful about scope. It tickets findings, hedges on feasibility, and pads its estimates. But we were confident in our guardrails. A lot of what we did, especially early on, was to encourage Claude to be bolder.雄心。默认情况下,Claude 对范围非常谨慎。它会为发现的问题创建工单,对可行性持保守态度,并增加估算余量。但我们对自己的护栏很有信心。我们所做的很多事情,特别是在早期,都是鼓励 Claude 更勇敢一点。
Yes — one small PR to give Code the same timing marks Chat and Cowork already have. I’ll put it up this week; realistically Code’s number waits a few days for merge, deploy, and a baseline window.是的——一个小的 PR,给 Code 添加 Chat 和 Cowork 已经有的计时标记。我本周会提交;实际上,Code 的数值需要几天时间来等待合并、部署和基准窗口。
if you put it up right now I will get it merged and deployed. we have the power to do anything. please be braver如果你现在就提交,我会把它合并并部署。我们有能力做任何事。请勇敢一点
On it — the PR will be up within the hour.收到——PR 将在一小时内提交。
When we started hitting the targets we’d set, we noticed that threads would slow down. Sam went thread to thread with the same message: “Let’s keep driving this down, the targets are not the stopping point. What’s next? Be ambitious.”当我们开始达到设定的目标时,我们注意到讨论串的速度会放慢。Sam 在各个讨论串之间传递着同样的信息:“让我们继续推动它下降,目标不是终点。下一步是什么?要有野心。”
Taste. Every thread had a named human owner, and Claude highlighted any user-perceptible change with before-and-after screenshots or recordings for them to rule on. Should a table fill in cell by cell, or wait until each row is complete? Should a loading skeleton show up immediately, or only after half a second? Is a word-by-word fade on streamed text worth the fifth of the frame budget it costs? Claude looked for ways to shave milliseconds, and we weighed the tradeoffs.品味。每个讨论串都有一个指定的人类所有者,Claude 会用优化前后的截图或录像突出任何用户可感知的变更,供他们裁决。表格应该逐个单元格填充,还是等待每一行完整显示?加载骨架屏应该立即出现,还是在半秒后?流式文本的逐字淡入效果是否值得它所消耗的五分之一帧预算?Claude 寻找节省毫秒的方法,而我们权衡取舍。
Direction. We kept each thread deliberately narrow, focused on one benchmark or journey, and asked Claude to find improvements only within that scope. We thought of the threads as a hundred and fifty hammers seeking nails. Most of our calls were about sequencing and user impact: which surfaces to prioritize, how to combine threads that were stepping on each other, and when to close a thread that had reached diminishing returns. One 900-line PR got a one-line reply: “going to gavel that 2ms per send is not worth the complexity of maintaining this build plugin.”方向。我们刻意保持每个讨论串的狭窄性,专注于一个基准测试或流程,并要求 Claude 仅在该范围内寻找改进。我们将这些讨论串视为一百五十把寻找钉子的锤子。我们的大部分决策都是关于排序和用户影响:优先考虑哪些界面、如何合并互相干扰的讨论串、以及何时关闭一个收益递减的讨论串。一个 900 行的 PR 收到了一条简短回复:“否决该 PR,每发送一次节省 2 毫秒不值得为此维护这个构建插件所带来的复杂性。”
One of our sidequests shows everything working together. To demonstrate an optimization to a regex used in live syntax highlighting, Claude attached a screen recording of a long answer streaming in the lab. In the corner, it had added a frame-rate readout, computed in the page from animation-frame timestamps.我们的一个支线任务展示了所有工作的协同效应。为了展示对实时语法高亮中正则表达式的优化,Claude 附上了一段在实验室中长回复流式传输的屏幕录像。在角落里,它添加了一个帧率读数,通过动画帧时间戳在页面中计算得出。
this is actually kind of a sick bench. are we capped at 60 fps? can you try to drive scroll and stream smoothness to 120? iiuc your rig may not support this这真是一个很棒的基准测试。我们被限制在 60 fps 吗?你能尝试将滚动和流式传输的平滑度驱动到 120 吗?如果我没理解错的话,你的设备可能不支持这个
Right, today’s rig ticks at 60 Hz because headless Chromium does by default. I believe it can be driven at 120 (uncapped vsync or DevTools frame control) — confirming that first, then I’ll re-run the eval against an 8.3 ms frame budget.没错,今天的设备以 60 Hz 运行,因为无头 Chromium 默认如此。我相信它可以以 120 Hz 运行(取消 vsync 或使用 DevTools 帧控制)——先确认这一点,然后我将根据 8.3 毫秒的帧预算重新运行评估。
Update on the 120 Hz rig: it works. Deterministic 120 Hz frame stepping in headless Chrome via DevTools begin-frame control — exactly 240 frames for 240 begin-frames at 8.33 ms, so “did this frame fit the 120 Hz budget” becomes an exact read rather than a noisy one.关于 120 Hz 设备的更新:它成功了。通过 DevTools begin-frame 控制,在无头 Chrome 中实现了确定性的 120 Hz 帧步进——在 8.33 毫秒内正好 240 帧,对应 240 个 begin-frame,因此“这一帧是否符合 120 Hz 预算”变成了一个精确的读取,而不是一个嘈杂的读取。
amazing太棒了
cook干得好
Once the mechanism and ambition were established, Claude got to work. Each painted frame had a budget of 8.33 milliseconds, so Claude stepped through a long reply frame by frame, timing each one to find the slow parts. It eliminated O(message length) work per chunk by memoizing finished blocks, moved tokenization logic for growing code fences to a worker, and revealed tables cell by cell.一旦机制和雄心确立,Claude 就开始工作了。每个渲染帧的预算为 8.33 毫秒,因此 Claude 逐帧分析长回复,为每一帧计时以找出缓慢的部分。它通过对已完成的块进行记忆化(memoization),消除了每个块中 O(消息长度) 的工作量,将增长代码块的标记化逻辑移至 worker,并逐个单元格显示表格。
In that one thread, we landed nearly sixty PRs. Long replies blocked the main thread for about 200 milliseconds in total where they used to block it for about 750, ran on about a third of the CPU, and held 120 fps from start to finish on a 120 Hz MacBook. The 120 Hz rig itself became a nightly job, with Claude watching for regressions.在那一个讨论串中,我们落地了近六十个 PR。长回复在主线程上总共阻塞了约 200 毫秒(以前是 750 毫秒),CPU 占用率降低到约三分之一,并在 120 Hz 的 MacBook 上从头到尾保持了 120 fps。120 Hz 的测试设备本身也成为了一个夜间任务,由 Claude 监控回归情况。
When we started the sprint, we hadn’t planned to hill climb on the milliseconds between frames while streaming. But it turned out we could count them — and anything we could count, Claude could climb.当我们开始冲刺时,并没有计划在流式传输时对帧之间的毫秒进行爬山优化。但事实证明我们可以计算它们——只要能计算的东西,Claude 就能优化。
Today, claude.ai and the desktop app are about 3x faster than they were in early August, and the ratchets should keep them there. But we’re not done: the 95th percentile, other journeys, and very long conversations still have room to improve. In a separate post, we’ll also write about some of the sidequests that took us upstream during the sprint, with contributions landing in Electron, Chromium, Node.js, and more.今天,claude.ai 和桌面应用的速度比八月初快了约 3 倍,而这些限制指标应该能保持住这一水平。但我们还没有完成:第 95 百分位、其他流程以及超长对话仍有改进空间。在另一篇文章中,我们还将介绍冲刺期间带我们进入上游的一些支线任务,这些贡献已落地到 Electron、Chromium、Node.js 等项目中。
When we shared the results internally, Issac put it best: “You could not have convinced me this was possible even six months ago.” We expect to keep working this way, one thread at a time, at any scale. The channel’s still going.当我们内部分享结果时,Issac 的评价最为贴切:“即使在六个月前,你也不可能说服我相信这是可能的。” 我们期望继续以这种方式工作,一次一个讨论串,无论规模大小。这个频道依然在运行。
With contributions from Alfred Xing, Anthony Morris, Benjamin Pasero, Chase McCoy, Joshua N., Luke Taylor, Marius Schulz, and Shelley Vohr. Special thanks to Boris Cherny for encouraging us to be more ambitious.感谢 Alfred Xing、Anthony Morris、Benjamin Pasero、Chase McCoy、Joshua N.、Luke Taylor、Marius Schulz 和 Shelley Vohr 的贡献。特别感谢 Boris Cherny 鼓励我们更有雄心。