Migrating the GitHub Copilot runtime to Rust, using Copilot

A rewrite this size wasn’t affordable before agents. Here’s what porting the Copilot agent runtime to 800,000 lines of production Rust actually took.

Copilot appears against a decorative background with scattered green squares.
| 65 minutes

The GitHub Copilot CLI, GitHub Copilot app, and GitHub Copilot SDK are all backed by the Copilot agent runtime, an agentic harness that can be embedded into applications and services. It was originally written in TypeScript on Node.js and the V8 JavaScript engine for what is now the GitHub Copilot cloud agent (CCA), and the runtime stayed on that stack as the runtime and its capabilities grew rapidly.GitHub Copilot CLI、GitHub Copilot 应用和 GitHub Copilot SDK 均由 Copilot 代理运行时(Copilot agent runtime)提供支持。这是一种可以嵌入到应用程序和服务中的代理工具。它最初是为现在的 GitHub Copilot 云代理(CCA)使用 TypeScript 在 Node.js 和 V8 JavaScript 引擎上编写的,随着运行时及其功能的快速增长,它一直保持在该技术栈上。

That has now changed. Using the GitHub Copilot app and the Copilot CLI, we completely rewrote the runtime into more than 800,000 lines of production Rust. AI agents wrote most of the code, spanning 128 pull requests that landed in main and shipped incrementally rather than waiting for a single cutover at the end. The few inevitable regressions were discovered and fixed quickly along the way, while the performance of the runtime improved by orders of magnitude. A project that would have taken a whole team of developers a year or two before agents was now completed primarily by a single developer, in only a few months, all while the rest of the team continued to greatly expand the runtime’s capabilities and reach.这种情况现在发生了改变。我们利用 GitHub Copilot 应用和 Copilot CLI,将运行时完全重写为超过 80 万行的生产级 Rust 代码。大部分代码由 AI 代理编写,跨越了 128 个合并到主分支的拉取请求(pull request),并以增量方式交付,而不是等待最后一次性的切换。沿途发现并快速修复了少数不可避免的回归问题,同时运行时的性能得到了数量级的提升。一个在 AI 代理出现前需要整个开发团队耗时一两年才能完成的项目,现在主要由一名开发人员在短短几个月内就完成了,而且在此期间,团队的其他成员还在持续大幅扩展该运行时的功能和覆盖范围。

Why we needed to port为什么我们需要进行移植

The Copilot agent runtime isn’t just the engine behind the Copilot CLI. It backs a growing set of Microsoft, GitHub, and ecosystem solutions, for each of which AI support is, architecturally, a shell around the same runtime plus whatever customizations that solution needs. This includes not only the GitHub Copilot CLI and the GitHub Copilot app, but also the latest releases of VS Code, Visual Studio, CCA, Copilot Code Review (CCR), Copilot Cowork, Copilot Studio, and Excel and Outlook and PowerPoint and Word and… it goes on.Copilot 代理运行时不仅仅是 Copilot CLI 背后的引擎。它为微软、GitHub 以及生态系统中越来越多的解决方案提供支持。在架构上,对于每一个解决方案,AI 支持都是围绕同一个运行时构建的“外壳”,并根据该解决方案的需求进行定制。这不仅包括 GitHub Copilot CLI 和 GitHub Copilot 应用,还包括 VS Code、Visual Studio、CCA、Copilot 代码审查(CCR)、Copilot Cowork、Copilot Studio 以及 Excel、Outlook、PowerPoint、Word 等产品的最新版本……不胜枚举。

These are very different products, and none of them wants to or should need to implement everything that goes into a production agent harness. They want all of the intelligence, security, reliability, and performance, and they want it shared so that a fix in one place fixes it in all of them. Most of the products listed in the previous paragraph initially implemented their own agent loop, but have since replaced it with the GitHub Copilot SDK, which is the entry point to the Copilot agent runtime. Doing so enables them to focus on their core business value and leave the details to the runtime. That’s all the more important given the pace of the industry and the employed agent loop needing to stay always best-of-breed in the face of intense competition.这些产品各不相同,它们都不希望也不应该需要实现生产级代理工具所需的一切功能。它们需要共享所有智能、安全性、可靠性和性能,以便在一个地方进行的修复能同时解决所有产品中的问题。上一段中列出的大多数产品最初都实现了自己的代理循环,但后来都将其替换为 GitHub Copilot SDK,这是进入 Copilot 代理运行时的入口点。这样做使它们能够专注于自身的核心业务价值,将细节留给运行时处理。考虑到行业的发展速度,以及所使用的代理循环必须在激烈的竞争中始终保持业界领先地位,这一点尤为重要。

So, shared runtime, good. The problem was the nature of the thing being shared.因此,共享运行时是件好事。问题在于所共享事物的本质。

If we look at the CLI, it’s logically a terminal UI (TUI) on top of an agent loop. As it happened, the whole stack was implemented in TypeScript, using Node.js as the framework and V8 for the execution engine, with Ink and React for UI. That’s a respectable choice for a TUI application; TypeScript and Node.js are broadly accessible and enable very rapid application development. And for the needs of a console application, the performance implications in terms of startup, responsiveness, throughput, and memory consumption are also reasonable. They are, unfortunately, much less reasonable when you think about that implementation being used in other environments, with other constraints, with demands for things like fast startup and excellent server density due to low memory overhead.如果我们看看 CLI,它在逻辑上是一个基于代理循环的终端 UI (TUI)。事实上,整个技术栈都是用 TypeScript 实现的,使用 Node.js 作为框架,V8 作为执行引擎,并使用 Ink 和 React 进行 UI 开发。对于 TUI 应用程序来说,这是一个不错的选择;TypeScript 和 Node.js 广泛易用,能够实现非常快速的应用程序开发。对于控制台应用程序的需求而言,在启动、响应速度、吞吐量和内存消耗方面的性能影响也是合理的。遗憾的是,当你考虑到这种实现被用于其他环境、面临其他约束,以及由于低内存开销而对快速启动和卓越服务器密度有要求时,这些性能影响就不那么合理了。

The architecture of the CLI and its runtime also contributed to challenges here. The whole industry is running extremely fast, and in that context, really bright people make decisions for delivery speed and market reach. The Copilot CLI was initially written and shipped quickly, and in doing so, the TUI and the runtime were fairly intertwined rather than separated into discrete layers. Then when an SDK was needed for programmatic access to that runtime, without clear separation of the layers, a pragmatic decision was made to layer the SDK on top of the CLI, even though logically you’d expect the inverse architecture. Rather than only being accessible via commands provided by the user at the command line, the CLI was updated with a mode where it could be run headless, reading similar commands from stdin and writing responses to stdout. A JSON-RPC protocol could then be used to marshal function calls from an external process to and from the CLI. The SDK could then be embedded in arbitrary consuming programs, which would spawn a CLI process to host the agent loop out-of-process, with the SDK calling functions in the remote process via this JSON-RPC mechanism. Neat. Fast to get out the door. Flexible. But not great for the performance (startup, memory, throughput) and reliability of those consuming applications. Creating a new CopilotClient from the SDK meant spawning another process:CLI 及其运行时的架构也导致了这里的挑战。整个行业发展极快,在这种背景下,真正聪明的人会为了交付速度和市场覆盖范围而做出决策。Copilot CLI 最初编写和发布得很快,因此 TUI 和运行时相当紧密地交织在一起,而不是被分离成独立的层。后来,当需要一个 SDK 来以编程方式访问该运行时,且缺乏清晰的层级分离时,我们做出了一个务实的决定:将 SDK 叠在 CLI 之上,尽管从逻辑上讲,你可能期望的是相反的架构。CLI 不仅可以通过用户在命令行提供的命令进行访问,还更新了一种模式:它可以在无头(headless)模式下运行,从 stdin 读取类似命令并将响应写入 stdout。然后,可以使用 JSON-RPC 协议将函数调用从外部进程编组到 CLI 并从中返回。这样,SDK 就可以嵌入到任意消费程序中,这些程序会生成一个 CLI 进程来在进程外托管代理循环,SDK 通过这种 JSON-RPC 机制调用远程进程中的函数。很巧妙。发布速度快。灵活。但对于那些消费应用程序的性能(启动、内存、吞吐量)和可靠性来说,这并不理想。从 SDK 创建一个新的 CopilotClient 意味着生成另一个进程:

const client = new CopilotClient();
await client.start(); // spawns the CLI as a subprocess
const session = await client.createSession({
    /* ... */
});

The process would need to launch and host Node and V8. It meant parsing the significant amount of JavaScript produced from the TypeScript code in the CLI, generating bytecode for it, and potentially optimizing hot code in later JIT tiers. It meant all the memory overhead associated with V8. It meant inheriting Node’s threading model, which by default pushes us towards a model of all CPU-bound work being serialized. And it meant forced out-of-process communication just to make function calls. It meant every SDK consumer, in every language, ships Node.js or a bundled binary containing V8. It meant the C#, Python, Go, Java, and Rust SDKs all paid for a whole second language runtime per client, on the order of 100 MB of working set minimum, for a runtime their application otherwise had no use for. It meant every event, every message, and every abstracted session file system read and write was pushed across a process boundary. It meant a crash in Node took the session with it. And it meant anyone deploying this had, at a minimum, two processes to supervise, monitor, and debug.该进程需要启动并托管 Node 和 V8。这意味着要解析 CLI 中由 TypeScript 代码生成的大量 JavaScript,为其生成字节码,并可能在后续的 JIT 层中优化热代码。这意味着与 V8 相关的所有内存开销。这意味着继承 Node 的线程模型,该模型默认将我们推向所有 CPU 密集型工作都被序列化的模型。而且这意味着仅仅为了进行函数调用,就必须进行强制的进程外通信。这意味着每一种语言的每个 SDK 消费者都要携带 Node.js 或包含 V8 的捆绑二进制文件。这意味着 C#、Python、Go、Java 和 Rust SDK 都必须为每个客户端支付第二个语言运行时的开销,其工作集至少在 100 MB 左右,而这些应用程序本身并不需要该运行时。这意味着每一个事件、每一条消息以及每一个抽象的会话文件系统读写操作都被推送到进程边界之外。这意味着 Node 中的崩溃会导致会话随之终止。而且这意味着任何部署此程序的人,至少需要监督、监控和调试两个进程。

Instead, we wanted a runtime:相反,我们想要一个运行时:

  • that does not include the TUI, that’s its own library the TUI and other applications and services can be properly layered on top of cleanly.不包含 TUI,TUI 和其他应用程序及服务可以清晰地分层构建在它之上。
  • implemented in a language with minimal dependencies and minimal overhead.用一种具有最小依赖和最小开销的语言实现。
  • implemented in a way that it can be cleanly embedded in-process rather than being forced out-of-process.以一种可以干净地嵌入进程内的方式实现,而不是被迫在进程外运行。
  • implemented in a language with top characteristics around performance and scalability and reliability.用一种在性能、可扩展性和可靠性方面具有顶级特性的语言实现。
  • implemented in a language that’s great for interop, such that it can be used cleanly by all six Copilot SDK language versions (C#, TypeScript, Python, Rust, Go, Java) with that stack’s foreign function interface (FFI) mechanism.用一种非常适合互操作的语言实现,这样它可以通过该技术栈的外部函数接口(FFI)机制被所有六个 Copilot SDK 语言版本(C#、TypeScript、Python、Rust、Go、Java)干净地使用。
  • implemented with a tool chain that provides a more modern security posture, with less supply chain risk and greater support for correct-by-construction code.使用一种提供更现代安全态势的工具链实现,具有更低的供应链风险和对“构造即正确”(correct-by-construction)代码的更好支持。

For all those reasons, as well as softer reasons (such as team experience and industry direction), we chose Rust. This is in no way a claim that every large TypeScript program should become Rust. Our requirements emphasized embedding through a C ABI, low startup and steady-state overhead, and predictable resource use. Rust made those goals possible, at the expense of other complications, e.g. we had to represent lifetimes and shared state explicitly (the lifecycle regressions discussed later highlight the implications of that). The right target language legitimately varies from application to application.基于所有这些原因,以及一些次要原因(例如团队经验和行业方向),我们选择了 Rust。这绝不是说每个大型 TypeScript 程序都应该变成 Rust。我们的需求强调通过 C ABI 进行嵌入、低启动和稳态开销以及可预测的资源使用。Rust 使这些目标成为可能,尽管付出了其他复杂的代价,例如我们必须显式地表示生命周期和共享状态(后面讨论的生命周期回归问题凸显了这一点的影响)。正确的目标语言确实因应用程序而异。

There were then two key related tasks undertaken:随后进行了两项关键的相关任务:

  1. Separating the TUI-specific code from the runtime, so that the former is layered strictly on top of the latter, and more specifically layered strictly on top of the SDK’s public surface area. Today, the CLI still calls directly into runtime internals in several places; moving it fully onto the SDK’s surface area is ongoing work.将 TUI 特定代码与运行时分离,使前者严格分层在后者之上,更具体地说,严格分层在 SDK 的公共表面之上。今天,CLI 在几个地方仍然直接调用运行时内部;将其完全迁移到 SDK 的表面上是一项正在进行的工作。
  2. Porting that runtime layer to 100% Rust, resulting in a pure native binary exposing a C ABI for in-process consumption by all the language front-ends and a stdin/stdout-based or socket-based server for when out-of-process is still desired.将该运行时层 100% 移植到 Rust,产生一个纯原生二进制文件,为所有语言前端提供用于进程内消费的 C ABI,并为仍然需要进程外运行的情况提供基于 stdin/stdout 或 socket 的服务器。

This post primarily covers the second: porting the runtime to Rust.本文主要介绍第二部分:将运行时移植到 Rust。

The Copilot runtime architecture before and after the Rust rewrite, showing the old SDK-to-CLI process boundary and the new in-process and out-of-process hosting paths.

What it looked like before移植前的样子

The initial porting plan in early May 2026 estimated the runtime at roughly 130,000 lines of TypeScript. For scoping purposes, this initial measurement was reasonably accurate, but, as it turned out, also wildly misleading, in two key ways. Concurrent with porting:2026 年 5 月初的初步移植计划估计运行时约为 13 万行 TypeScript 代码。出于范围界定的目的,这一初步测量相当准确,但事实证明,它在两个关键方面也极具误导性。在移植的同时:

  1. Pieces still wrapped up in the TUI layer were being pushed down to the runtime layer. Entire components and significant percentages of code initially ignored in the estimates were then later considered relevant to porting.仍包裹在 TUI 层中的部分被下推到运行时层。最初在估算中被忽略的整个组件和很大比例的代码后来被认为与移植相关。
  2. Pull requests contributing significant amounts of new TypeScript were constantly raising the amount of TypeScript in the repo. Tens of agentically assisted developers merging hundreds of pull requests per week.贡献大量新 TypeScript 代码的拉取请求不断增加仓库中的 TypeScript 数量。数十名由代理辅助的开发人员每周合并数百个拉取请求。

Everything factored in, I estimate approximately 430,000 lines of production TypeScript ended up passing through the port. Those same factors also made it hard to see progress along the way: until close to the end, production TypeScript volume appeared to be holding relatively steady, if not increasing slightly, as porting kept pace with incoming work.综合考虑所有因素,我估计最终通过移植的生产级 TypeScript 代码大约有 43 万行。同样的因素也使得在过程中很难看到进展:直到接近尾声时,生产级 TypeScript 的总量看起来相对稳定,甚至略有增加,因为移植速度与传入的工作量保持同步。

This is confused further because there was also incoming Rust code, separate from the port, over the timeframe; early in the porting effort, incoming code was more likely to be dominated by TypeScript, whereas later in the effort, it was more likely to be dominated by Rust.这一点更加令人困惑,因为在此期间还有与移植无关的传入 Rust 代码;在移植工作的早期,传入的代码更有可能以 TypeScript 为主,而在工作后期,则更有可能以 Rust 为主。

TypeScript fell to zero while production Rust rose to approximately 830,000 lines and Rust unit tests to approximately 469,000 lines.

During the port, the runtime took in ~300,000 production lines of TypeScript and shed ~430,000, while ~1,200,000 production Rust lines entered and ~365,000 left. In other words, the apparent stability of the TypeScript line in the above graph was actually hiding significant amounts of TypeScript churn.在移植期间,运行时引入了约 30 万行生产级 TypeScript 代码并移除了约 43 万行,同时引入了约 120 万行生产级 Rust 代码并移除了约 36.5 万行。换句话说,上图中 TypeScript 代码行数的表面稳定性实际上掩盖了大量的 TypeScript 代码更迭。

In-place porting strategy就地移植策略

That chart also highlights an important aspect of how the port was done: in place.该图表还强调了移植方式的一个重要方面:就地(in place)移植。

There are two main approaches to a rewrite of this scale:这种规模的重写主要有两种方法:

  1. Big bang. The new Rust runtime is developed as a complete alternative and then swapped in all at once when it’s ready. Such a big-bang cutover has two variations. a. Stop the world. Everyone ceases other work on the main branch while the rewrite happens, with the rewrite being done in main. b. Parallel development. The rewrite happens in a feature branch while work continues in the main branch, with the rewrite constantly trying to keep up with and merging in changes from the main branch.大爆炸式(Big bang)。新的 Rust 运行时被开发为一个完整的替代品,然后在准备好后一次性全部替换。这种大爆炸式切换有两种变体:a. 停止世界(Stop the world)。在重写期间,每个人都停止在 main 分支上的其他工作,重写在 main 分支中进行。b. 并行开发。重写在功能分支中进行,同时在 main 分支中继续工作,重写不断尝试跟上并合并来自 main 分支的更改。
  2. In place. This is done as a component-by-component port, where the runtime is incrementally rewritten one piece at a time. Such an in-place approach also has two variations. a. Atomic replacement. Each piece is flipped atomically from TypeScript to Rust, with interop between the remaining TypeScript and the new Rust providing continuity. Over time, less and less of the production runtime is TypeScript, and more and more is Rust, until one day, there’s no more TypeScript, only Rust. b. A/B. Rather than deleting components as they’re ported, both the TypeScript and the Rust components are maintained as hot-swappable options, with the TypeScript being deleted once confidence has plateaued.就地(In place)。这是作为组件级移植完成的,运行时被逐个部分地增量重写。这种就地方法也有两种变体:a. 原子替换。每个部分都从 TypeScript 原子地切换到 Rust,剩余的 TypeScript 和新的 Rust 之间的互操作性提供了连续性。随着时间的推移,生产运行时的 TypeScript 越来越少,Rust 越来越多,直到有一天,不再有 TypeScript,只有 Rust。b. A/B 测试。与其在移植时删除组件,不如将 TypeScript 和 Rust 组件都作为可热插拔的选项进行维护,在信心达到稳定水平后删除 TypeScript。

We went with option 2a, for a variety of reasons:我们选择了方案 2a,原因有多种:

  • No one experiences work stoppage. The main branch continues to be active. Every developer not directly involved in the port gets to keep on keepin’ on, impacted only when a pull request they may have in flight for a prolonged period of time happens to touch code that gets ported concurrently, in which case they need to rebase and have their agents help port just their in-flight changes.没有人经历工作停顿。main 分支保持活跃。每个未直接参与移植的开发人员都可以继续工作,只有当他们正在进行的长周期拉取请求恰好触及同时被移植的代码时才会受到影响,在这种情况下,他们需要重新基准化(rebase)并让他们的代理帮助移植他们正在进行的代码更改。
  • The runtime’s main branch is always shippable. Each pull request replaces the existing TypeScript implementation with a thin shim that calls into Rust, and deletes the old code in one atomic change. The new code is immediately exercised, in-situ.运行时的 main 分支始终是可发布的。每个拉取请求都用一个调用 Rust 的轻量级垫片(shim)替换现有的 TypeScript 实现,并在一个原子更改中删除旧代码。新代码立即在原位(in-situ)得到执行。
  • The rewrite is incremental and reviewable. Each pull request ports a single component or slice, so the scope of change is smaller and the diff is easier to review, whether by a human or by agents or both.重写是增量且可审查的。每个拉取请求都移植一个组件或片段,因此更改范围更小,差异更容易审查,无论是人工审查、代理审查还是两者兼而有之。
  • Most ports are reasonably small and self-contained, minimizing drift from concurrent pull requests. In some cases, where TypeScript components were too large, they could first be refactored into more easily ported components.大多数移植都相当小且自包含,最大限度地减少了并发拉取请求带来的漂移。在某些情况下,如果 TypeScript 组件太大,可以先将其重构为更容易移植的组件。
  • All existing end-to-end tests, across the CLI and SDK, run against the new Rust code at every step, giving us confidence and lots of validation. If a pull request caused a required test to fail, it didn’t land.所有现有的端到端测试(跨 CLI 和 SDK)在每一步都针对新的 Rust 代码运行,这给了我们信心和大量的验证。如果一个拉取请求导致必要的测试失败,它就不会被合并。

We also shied away from the option 2b variant that involved maintaining multiple versions of the same component concurrently. With hundreds of pull requests being merged into the repo per week for the last several months, the codebase is constantly evolving, and quickly. Having two different versions of the same code in two different languages and using two different sets of dependency libraries adds a ton of complexity. Some of these components aren’t perfectly isolated, either; while some are logically standalone with simple APIs for accessing them by the rest of the system, others have significant tendrils, and making that graph hot swappable per component is a nightmare. The subsystems that would possibly benefit the most from a cautious parallel cutover are exactly the ones where parallel is hardest. For example, session orchestration isn’t a pure function you can call two different versions of with an if/else on some experimentation flag. It owns mutable state, drives callbacks in both directions, and threads through nearly every other subsystem, so “run both and compare” would mean maintaining two divergent copies of the component that holds the conversation’s state and services, and praying they stay in sync across hundreds of concurrent edits. The coupling that makes a component hard to port is the same coupling that makes it near impossible to shadow without risking introducing more regressions than it avoids. The benefits of being able to swap in this manner are primarily about gaining confidence, which we could do in other ways.我们也避开了方案 2b 中涉及同时维护同一组件的多个版本的变体。在过去几个月中,每周有数百个拉取请求被合并到仓库中,代码库在不断且快速地演进。在两种不同的语言中使用两种不同版本的代码,并使用两套不同的依赖库,会增加巨大的复杂性。其中一些组件也不是完全隔离的;虽然有些在逻辑上是独立的,并具有简单的 API 供系统其余部分访问,但另一些则有大量的纠缠,使该图表在每个组件上实现热插拔简直是一场噩梦。那些可能从谨慎的并行切换中获益最多的子系统恰恰是并行化最困难的。例如,会话编排(session orchestration)不是一个你可以通过某种实验标志使用 if/else 调用两个不同版本的纯函数。它拥有可变状态,在两个方向上驱动回调,并穿插在几乎所有其他子系统中,因此“同时运行并比较”意味着维护持有对话状态和服务的组件的两个不同副本,并祈祷它们在数百个并发编辑中保持同步。使得组件难以移植的耦合,同样使得在不冒引入更多回归风险的情况下对其进行影子化(shadowing)几乎是不可能的。以这种方式进行交换的好处主要在于获得信心,而我们有其他方法可以做到这一点。

Validation also happened through incremental rollout. With a big-bang cutover approach, we would hold everything in a long-lived branch, port the whole runtime, and cut over once. That means consumers experience every ported line at once, including all regressions that slipped through in-repo testing. Rolling out portions of the change incrementally, two components here, one component there, enabled us to get a last-mile of validation in deployed builds with real consumer usage (most often first-party within Microsoft and GitHub) but while keeping the risk of regression to a minimum. Over the roughly fourteen-and-a-half-week porting window, main shipped 135 releases, inclusive of 100 pre-release versions and 35 stable versions, averaging around 1.3 releases per day. Roughly 1.3 port pull requests opened per day, as well, such that each release carried a small and knowable set of ported components (we generally tried but didn’t always succeed in shipping ports in a pre-release first). In a trailing seven-day npm sample, pre-release versions accounted for only 10.5% of downloads, indicating that initial exposure was relatively limited while we monitored feedback channels for signals of things breaking and quickly turned around fixes in the next pre-release. Reported issues were more easily correlated with known recent changes, and more easily root caused and quickly fixed. In this manner, doing the porting incrementally over a longer period of time was actually a feature rather than a hindrance (i.e. faster is not always better). By August 21, the runtime was 100% production Rust: 832,378 lines of production Rust and 468,689 lines of Rust unit tests, in addition to 174,675 lines of E2E TypeScript tests. The separate GitHub Copilot SDK repository added another ~130,000 lines of E2E test code across Node.js, Python, Go, C#, Rust, and Java.验证也通过增量发布实现。采用大爆炸式切换方法,我们会将所有内容保留在一个长期的分支中,移植整个运行时,然后一次性切换。这意味着消费者会同时体验到每一行移植的代码,包括在仓库内测试中漏掉的所有回归问题。将更改的部分增量发布,这里两个组件,那里一个组件,使我们能够在部署版本中获得真实的消费者使用验证(通常是微软和 GitHub 内部的第一方使用),同时将回归风险降至最低。在约 14.5 周的移植窗口期内,main 分支发布了 135 个版本,包括 100 个预发布版本和 35 个稳定版本,平均每天约 1.3 个版本。大约每天也有 1.3 个移植拉取请求打开,因此每个版本都携带了一小部分已知且可控的已移植组件(我们通常尝试但不总是成功地先在预发布版本中发布移植)。在最近七天的 npm 样本中,预发布版本仅占下载量的 10.5%,这表明在监控反馈渠道以获取故障信号并在下一个预发布版本中快速修复时,初始暴露范围相对有限。报告的问题更容易与最近已知的更改相关联,也更容易找到根本原因并快速修复。通过这种方式,在较长时期内增量进行移植实际上是一个特性而不是障碍(即:更快并不总是更好)。到 8 月 21 日,运行时已 100% 为生产级 Rust:包含 832,378 行生产级 Rust 代码和 468,689 行 Rust 单元测试,此外还有 174,675 行 E2E TypeScript 测试。独立的 GitHub Copilot SDK 仓库在 Node.js、Python、Go、C#、Rust 和 Java 中又增加了约 13 万行 E2E 测试代码。

Timeline from May 12 through August 21 showing 128 port pull request merges sized by changed lines and 135 public CLI releases; the largest port changes cluster near completion of the port.

Getting going开始行动

Before going all in, we gained confidence and proved things out. We started with two pull requests that established the Rust workspace, toolchain, lint rules, CI, build pipeline, and coding instructions, and then introduced the runtime crate plus code generation and interop patterns while porting a collection of pure-logic primitives chosen specifically because they had no I/O or shared state and already had strong tests. Only after those landed did the first primary port pull request take three side-effect-free helpers through the full process. These functioned as shipping pilots, turning assumptions about repository layout, FFI, packaging, testing, and review into conventions the subsequent much larger ports would then reuse. Basically, we tested the machinery end to end. The plan continued by ordering the work from the leaves inward, with pure helpers, content exclusion, shell utilities, and session filesystem operations establishing the translation and testing pattern. Stateful subsystems followed, and tools, hooks, model clients, and MCP built on those pieces. Session orchestration (by far the most coupled and least naturally parallel part of the runtime) would come near the end.在全力投入之前,我们获得了信心并证明了可行性。我们从两个拉取请求开始,建立了 Rust 工作区、工具链、lint 规则、CI、构建流水线和编码说明,然后引入了运行时 crate 以及代码生成和互操作模式,同时移植了一组纯逻辑原语,选择它们是因为它们没有 I/O 或共享状态,并且已经有了强大的测试。只有在这些落地后,第一个主要的移植拉取请求才将三个无副作用的辅助函数通过了整个过程。这些充当了发布试点,将关于仓库布局、FFI、打包、测试和审查的假设转化为后续更大规模移植将重用的约定。基本上,我们端到端地测试了这套机制。计划继续按照从叶子向内的顺序进行工作,通过纯辅助函数、内容排除、shell 工具和会话文件系统操作来建立翻译和测试模式。有状态的子系统紧随其后,工具、钩子、模型客户端和 MCP 建立在这些部分之上。会话编排(运行中最耦合且最不自然并行化的部分)将在最后阶段进行。

Timeline of 128 landed port pull requests from May through August, progressing from small foundational components to larger orchestration and session work.
PeriodPull requestsMedian changed lines
May 1–1583,250
May 16–3129,421
Jun 1–15405,073
Jun 16–30318,253
Jul 1–15109,514
Jul 16–311428,159
Aug 1–151913,861
Aug 16–30499,445

The early ports, small leaf components, moved quickly. But larger subsystems didn’t move in one atomic step; MCP support, for example, progressed through seven dedicated pull requests, while tools progressed via a six-part series and then needed additional work to move orchestration and retire the remaining TypeScript. Hooks, auth, telemetry, plugins, settings, and persistence followed similar paths.早期的移植,即小的叶子组件,进展迅速。但较大的子系统并没有通过一个原子步骤完成;例如,MCP 支持通过七个专门的拉取请求推进,而工具通过一个六部分系列推进,然后需要额外的工作来移动编排并淘汰剩余的 TypeScript。钩子、身份验证、遥测、插件、设置和持久化也遵循了类似的路径。

In effect, the useful unit of porting wasn’t always “a component.” It was often a wave through regions of related behavior: first move the pure logic, then move state ownership, then move orchestration, then remove fallbacks, and finally simplify the Rust after the temporary interop was gone.实际上,移植的有效单位并不总是“一个组件”。它通常是一波跨越相关行为区域的浪潮:先移动纯逻辑,然后移动状态所有权,然后移动编排,然后移除后备方案,最后在临时互操作消失后简化 Rust 代码。

Interop互操作

There are two main layers in this port involving interop:在这个移植中涉及两个主要的互操作层:

  1. Temporary internal interop. Any time a function was ported to Rust, that function needed to be invocable from whatever TypeScript was calling the initial TypeScript function. Similarly we needed to enable Rust functions to invoke TypeScript callbacks. This interop need is an implementation detail and extremely fluid. As the Rust internal surface area grows, so too do the number of TypeScript shims needed, as they’re 1:1 with whatever Rust methods need to be called from TypeScript. As those callers get ported to Rust, that existing layer of shim is deleted, and a new layer is put in place. Eventually we reach the public entrypoints into the runtime library, and the shims evaporate.临时内部互操作。每当一个函数被移植到 Rust 时,该函数都需要能够被调用原始 TypeScript 函数的任何 TypeScript 代码所调用。同样,我们需要使 Rust 函数能够调用 TypeScript 回调。这种互操作需求是一个实现细节,且极其易变。随着 Rust 内部表面的增长,所需的 TypeScript 垫片数量也在增加,因为它们与需要从 TypeScript 调用的 Rust 方法是一一对应的。随着这些调用者被移植到 Rust,现有的垫片层就会被删除,并放入一个新的层。最终我们到达了运行时库的公共入口点,垫片就会蒸发。
  2. The SDK surface. All of the SDK libraries need to be able to sit on top of the runtime and expose its functionality. In the pre-port world, this was done by having the runtime exposed via a bidirectional JSON-RPC layer, with the SDK sending function call requests as JSON-RPC method call payloads, the runtime parsing the request and invoking the relevant API, then sending back the result via the same transport for the SDK to parse and return. The inverse direction also exists; the runtime needs to be able to call back to the SDK client, for example for hook notifications and permission demands, which surface in the SDK clients as callbacks using whatever language feature is considered idiomatic (e.g. delegates in C#).SDK 表面。所有的 SDK 库都需要能够位于运行时之上并公开其功能。在移植前的世界里,这是通过通过双向 JSON-RPC 层公开运行时来完成的,SDK 将函数调用请求作为 JSON-RPC 方法调用负载发送,运行时解析请求并调用相关的 API,然后通过相同的传输方式发送回结果供 SDK 解析并返回。反向也存在;运行时需要能够回调 SDK 客户端,例如用于钩子通知和权限请求,这些在 SDK 客户端中表现为使用任何被认为是惯用语言特性(例如 C# 中的委托)的回调。

We achieved (1) via the napi Rust crate from the napi-rs project, which exists to build Node native addons in Rust. You annotate a function with #[napi], and a napi-rs macro generates the N-API registration glue that makes the function callable from JavaScript, plus a TypeScript declaration for it in a generated index.d.ts. A synchronous Rust function becomes an ordinary JavaScript function, an async fn becomes a JavaScript function returning a promise, and structs annotated #[napi(object)] become plain objects on the other side.我们通过 napi-rs 项目中的 napi Rust crate 实现了 (1),该 crate 旨在用 Rust 构建 Node 原生插件。你用 #[napi] 注解一个函数,napi-rs 宏就会生成使该函数可从 JavaScript 调用的 N-API 注册胶水代码,以及生成的 index.d.ts 中的 TypeScript 声明。同步的 Rust 函数变成普通的 JavaScript 函数,async fn 变成返回 promise 的 JavaScript 函数,而注解为 #[napi(object)] 的结构体在另一端变成普通对象。

Traffic has to move in both directions, too. Many ported components temporarily depended on something that hadn’t been ported yet, so Rust needed to call back into TypeScript, for example a tool implementation in Rust asking the still-TypeScript model layer for inference, or raising a hook, or requesting a permission decision for a command it wanted to run. napi-rs handles this with “threadsafe functions,” which let Rust code running on a Tokio worker thread invoke a JavaScript callback back on Node’s main thread. Node installs the callback once, Rust holds it and calls it whenever it needs to go the other way. Every one of these is temporary by construction: the callback exists only because the thing on the other end is still TypeScript, and it gets deleted when that thing is ported.流量也必须双向移动。许多已移植的组件暂时依赖于尚未移植的内容,因此 Rust 需要回调 TypeScript,例如 Rust 中的工具实现询问仍然是 TypeScript 的模型层进行推理,或者引发钩子,或请求权限决策以运行它想要的命令。napi-rs 通过“线程安全函数”处理这个问题,这允许在 Tokio 工作线程上运行的 Rust 代码调用回到 Node 主线程上的 JavaScript 回调。Node 安装一次回调,Rust 持有它并在需要反向操作时调用它。每一个这样的回调在设计上都是临时的:回调的存在仅仅是因为另一端的东西仍然是 TypeScript,当那个东西被移植时,它就会被删除。

Temporary Rust N-API exports and their TypeScript call sites grew during the incremental port, then declined as callers moved to Rust and the temporary interop surface disappeared.

The temporary seam peaked on August 3, with 2,019 internal N-API exports and 3,356 TypeScript call sites. At completion, the runtime was entirely Rust, therefore no internal interop: 0 temporary internal N-API exports and 0 TypeScript call sites remained. (I mentioned earlier that the CLI still has some internal access to the runtime we’re working to remove; those exports aren’t counted here.)这种临时缝合在 8 月 3 日达到顶峰,有 2,019 个内部 N-API 导出和 3,356 个 TypeScript 调用点。完成时,运行时完全是 Rust,因此没有内部互操作:剩余 0 个临时内部 N-API 导出和 0 个 TypeScript 调用点。(我之前提到 CLI 仍然有一些我们正在努力移除的对运行时的内部访问;这些导出不计入此处。)

The second interop layer, the SDK surface, is the permanent one of the two. The Copilot SDK ships for six languages: TypeScript, Python, Go, C#, Java, and Rust. All of them speak the same bidirectional JSON-RPC contract, and originally all of them reached it the same way: spawn the Copilot CLI in headless mode as a subprocess, and talk to it over a pipe or a socket. That remained the default during the port. It also meant an SDK consumer in any language ships or locates a full Node implementation, pays a process hop on every event and every message, and supervises two processes instead of one.第二个互操作层,即 SDK 表面,是两者中永久性的一个。Copilot SDK 为六种语言发布:TypeScript、Python、Go、C#、Java 和 Rust。它们都遵循相同的双向 JSON-RPC 合约,最初它们都以相同的方式达到该合约:以无头模式将 Copilot CLI 作为子进程生成,并通过管道或 socket 与其通信。这在移植期间仍然是默认设置。这也意味着任何语言的 SDK 消费者都要携带或定位一个完整的 Node 实现,在每个事件和每条消息上都要支付进程跳转的代价,并监督两个进程而不是一个。

Porting the runtime to Rust is what makes the other option viable. The shipped runtime.node is an ordinary platform shared library (the .node extension is the Node.js native-addon convention; underneath it is a .dll, .so, or .dylib), and it now presents two front doors onto the same engine. There’s the napi door, which a Node process loads as a native addon; that’s the CLI’s path (today… in the future, the intent is it’ll go through the SDK path fully). And there’s a C ABI door, which any language can load into its own process and call through FFI. The same in-process runtime is selected through each language’s native interop mechanism:将运行时移植到 Rust 使得另一个选项变得可行。已发布的 runtime.node 是一个普通平台的共享库(.node 扩展名是 Node.js 原生插件约定;其底层是一个 .dll、.so 或 .dylib),它现在为同一个引擎提供了两扇前门。一个是 napi 门,Node 进程将其作为原生插件加载;这是 CLI 的路径(今天……未来,意图是它将完全通过 SDK 路径)。另一个是 C ABI 门,任何语言都可以将其加载到自己的进程中并通过 FFI 调用。同一个进程内运行时通过每种语言的原生互操作机制进行选择:

SDKNative bridgeIn-process client selection
C#P/Invokenew CopilotClient(new CopilotClientOptions { Connection = RuntimeConnection.ForInProcess() })
Gopuregocopilot.NewClient(&copilot.ClientOptions{Connection: copilot.InProcessConnection{}})
JavaJNAnew CopilotClient(new CopilotClientOptions().setConnection(RuntimeConnection.forInProcess()))
PythoncffiCopilotClient(connection=RuntimeConnection.for_inprocess())
RustlibloadingClient::start(ClientOptions::new().with_transport(Transport::InProcess)).await?
TypeScriptkoffinew CopilotClient({ connection: RuntimeConnection.forInProcess() })

The Rust rewrite and the choice between in-process and out-of-process hosting are separate dimensions. The completed Rust runtime supports both: it can run inside the SDK consumer’s process or behind the existing JSON-RPC server boundary. Those in-process entry points are currently opt-in while we gain confidence in sharing a process, and therefore a failure boundary, with the consuming application. Everything above the transport remains the same SDK API: sessions, events, tools, permissions, and callbacks do not care whether their JSON-RPC bytes crossed a pipe or a function call.Rust 重写以及在进程内和进程外托管之间的选择是两个独立的维度。已完成的 Rust 运行时支持两者:它可以在 SDK 消费者的进程内运行,也可以在现有的 JSON-RPC 服务器边界后运行。这些进程内入口点目前是可选的,因为我们正在获得在共享进程(以及因此共享故障边界)中与消费应用程序共存的信心。传输之上的所有内容仍然是相同的 SDK API:会话、事件、工具、权限和回调并不关心它们的 JSON-RPC 字节是穿过了管道还是函数调用。

The interesting thing about the second door is its size. It has just 19 exported functions: four for server lifecycle, four for session registration and configuration, eight for connections, and three for the embedded host. Behind those functions, the shared contract currently contains 364 dispatch routes: 340 are callable by SDK consumers, while 24 run in the other direction as runtime-to-SDK callbacks. The napi door is much larger, needing functions for every one of those dispatch routes. The C ABI door is dispatch-based: API methods don’t get exports at all, but rather travel as JSON-RPC bytes written into a connection, and results, events, and server-to-client requests come back on host-supplied callbacks. Adding, changing, or removing an API method touches the engine’s dispatch table but never touches the ABI. An SDK binds those 19 entry points once and reaches the entire, still-growing API surface dynamically through them.关于第二扇门的有趣之处在于它的大小。它只有 19 个导出函数:四个用于服务器生命周期,四个用于会话注册和配置,八个用于连接,三个用于嵌入式宿主。在这些函数背后,共享合约目前包含 364 个分发路由:340 个可由 SDK 消费者调用,24 个作为运行时到 SDK 的回调在相反方向运行。napi 门要大得多,需要为每一个分发路由提供函数。C ABI 门是基于分发的:API 方法根本不获得导出,而是作为写入连接的 JSON-RPC 字节进行传输,结果、事件和服务器到客户端的请求通过宿主提供的回调返回。添加、更改或删除 API 方法会触及引擎的分发表,但永远不会触及 ABI。SDK 绑定这 19 个入口点一次,并通过它们动态地到达整个仍在增长的 API 表面。

Which raises the obvious question: why is there still JSON-RPC in a call that no longer crosses a process boundary?这就引出了一个显而易见的问题:为什么在不再跨越进程边界的调用中仍然存在 JSON-RPC?

The answer is that it made in-process hosting a drop-in rather than a rewrite. Every SDK already had a working JSON-RPC client, with framing, request and response correlation, and handlers for the server-to-client direction. Mounting FFI as one more transport underneath that client moves the byte path from a pipe or socket to a function call and leaves everything above it untouched. Six SDKs got in-process hosting as an additive, opt-in transport, with the existing ones unchanged. Had we instead defined a typed C function per API method, every SDK would have needed a second binding layer, every new API method would have needed six more bindings, and the ABI would have become a binary compatibility surface we had to version.答案是,这使得进程内托管成为一种直接插入(drop-in)方式,而不是重写。每个 SDK 已经拥有一个工作的 JSON-RPC 客户端,具有帧处理、请求和响应关联以及服务器到客户端方向的处理程序。将 FFI 作为该客户端之下的另一个传输方式,将字节路径从管道或 socket 移动到函数调用,并使之上的所有内容保持不变。六个 SDK 获得了进程内托管作为一种附加的、可选的传输方式,现有的方式保持不变。如果我们改为为每个 API 方法定义一个类型化的 C 函数,每个 SDK 都需要一个第二绑定层,每个新的 API 方法都需要多六个绑定,并且 ABI 将成为我们必须进行版本控制的二进制兼容性表面。

We also still need JSON-RPC for runtimes that genuinely are remote, whether across a subprocess boundary or over TCP. Keeping the same protocol in-process means maintaining one bidirectional API and dispatch system rather than JSON-RPC for remote connections plus a second per-method FFI surface for local ones. That’s a real tradeoff rather than a no-brainer decision. We avoid the process hop, but we still pay JSON-RPC overhead on every call. For inference-dominated workloads, that serialization is generally small potatoes compared to the model round trip. It is still measurable in high-throughput local workloads, but not enough today to justify duplicating hundreds of methods across six SDK bindings. And it’s a decision we can easily revise later should the performance need present itself. The payload encoding is a private detail of the two ends, and swapping JSON for something denser like MessagePack would not change a single declared export. Typed per-method exports can also be added later for hot paths, calling the same engine and the same handlers, without replacing the byte channel, which would remain the substrate for streaming, server-to-client requests, and the long tail of rarely called methods where a bespoke export buys nothing.对于真正远程的运行时,无论是跨越子进程边界还是通过 TCP,我们也仍然需要 JSON-RPC。在进程内保持相同的协议意味着维护一个双向 API 和分发系统,而不是为远程连接使用 JSON-RPC,为本地连接使用第二个按方法划分的 FFI 表面。这是一个真正的权衡,而不是一个不言而喻的决定。我们避免了进程跳转,但我们仍然在每次调用上支付 JSON-RPC 开销。对于以推理为主的工作负载,这种序列化通常与模型往返相比是小巫见大巫。它在高吞吐量的本地工作负载中仍然是可以测量的,但目前还不足以证明在六个 SDK 绑定中复制数百个方法的合理性。如果未来出现性能需求,我们可以很容易地修改这个决定。有效载荷编码是两端的私有细节,将 JSON 替换为像 MessagePack 这样更紧凑的东西不会改变任何声明的导出。类型化的按方法导出也可以在以后为热路径添加,调用相同的引擎和相同的处理程序,而无需替换字节通道,字节通道将仍然是流式传输、服务器到客户端请求以及很少调用的方法长尾的基质,对于这些方法,定制导出毫无意义。

What the session data shows会话数据显示了什么

Nearly every figure in this post comes from one of two sources. The first source is the GitHub history of the private github/copilot-agent-runtime repository: pull requests and their diffs, review comments, CI runs, etc. The second source is the agent session logs. The runtime (and thus the CLI, app, etc.) writes a structured event log for every session it runs: one JSON object per line, appended as the session happens. Those logs can include prompts, commands, command output, file paths, and potentially secrets surfaced by tools, so they must be handled as sensitive data. The log is local to the machine where the session ran; remote-session features can also upload it when enabled, subject to product settings and organizational policy.本文中几乎每个数字都来自两个来源之一。第一个来源是私有 github/copilot-agent-runtime 仓库的 GitHub 历史记录:拉取请求及其差异、审查评论、CI 运行等。第二个来源是代理会话日志。运行时(以及 CLI、应用等)为它运行的每个会话编写结构化事件日志:每行一个 JSON 对象,随着会话发生而追加。这些日志可能包含提示、命令、命令输出、文件路径以及工具可能暴露的潜在机密,因此它们必须作为敏感数据处理。日志位于会话运行的机器本地;启用后,远程会话功能也可以上传它,具体取决于产品设置和组织政策。

Here is a summary of the data across all the constituent porting pull requests:以下是所有构成移植的拉取请求中的数据摘要:

MetricCount
Events12,760,995
User messages31,247
Assistant messages1,385,214
Hook start and end events6,438,562
Tool starts1,857,409
Compilation commands23,096
Test commands19,485
Rebase commands2,496
Commit commands7,410
Push commands5,554
Completed compactions5,116

Those 31,247 user-role messages aren’t 31,247 prompts I personally typed; they include skill instructions, automated merge ticks, cross-session messages, and child-agent traffic, beyond the roughly 2,600 I typed or spoke, about one in 12. Similarly, the 1,385,214 assistant messages include subagents and tool-oriented messages, not just text shown to me in a conversational UI. The corpus contains 68 distinct event types and 67 distinct tool names; 1,130,921 tool calls, 61%, came from subagents rather than the main session thread.那 31,247 条用户角色消息并不是我个人输入的 31,247 条提示;它们包括技能指令、自动合并标记、跨会话消息和子代理流量,超出了我输入或口述的大约 2,600 条,大约每 12 条中有一条。同样,1,385,214 条助手消息包括子代理和面向工具的消息,而不仅仅是对话 UI 中显示给我的文本。语料库包含 68 种不同的事件类型和 67 种不同的工具名称;1,130,921 次工具调用(61%)来自子代理,而不是主会话线程。

The count doesn’t say why I was inserting myself ~2,600 times. For that, I had Copilot assign one primary intent to each of the human-authored messages in the session log corpus.这个计数并没有说明我为什么要介入约 2,600 次。为此,我让 Copilot 为会话日志语料库中每条人类编写的消息分配一个主要意图。

Of 2,639 human-authored messages, 31.0% focused on review, testing, and CI; 17.4% challenged technical or design decisions; and 15.0% pushed for completeness.

The first three buckets account for 63% of my interactions. Only ~40 were recognizable session kickoffs from me, as that largely happened by me first creating a chat to explore the next horizon and then asking that chat session to create actual porting sessions for each desired slice. My role was less “assign a task and wait” and more “operate the control loop”: inspect the result, challenge technical decisions, enforce quality gates, and push when an agent treated an intermediate stopping point as the finish line. Human judgment was still very heavily involved even though agents were doing “the work.” My involvement just moved upward… instead of being responsible for writing syntax, I was responsible for framing the problems, defining boundaries, choosing strategies, adjudicating exceptions, and overall ensuring everything was moving in a good direction.前三个桶占了我交互的 63%。只有约 40 次是我可识别的会话启动,因为这主要是我先创建一个聊天来探索下一个视野,然后要求该聊天会话为每个想要的切片创建实际的移植会话。我的角色与其说是“分配任务并等待”,不如说是“操作控制循环”:检查结果、挑战技术决策、强制执行质量门,并在代理将中间停止点视为终点时进行推动。即使代理在做“工作”,人类的判断仍然非常重要。我的参与只是向上移动了……我不再负责编写语法,而是负责构建问题、定义边界、选择策略、裁决异常,并总体上确保一切都朝着好的方向发展。

It’s all about caching一切都在于缓存

LLM providers typically charge one rate for input tokens (what you send to them) and another rate for output tokens (what they send to you). Billing is often done by tokens because they are a useful approximation of the computational work required to do the inference: for each input token, the model must read it, incorporate it into its internal representation, and use it as part of the computation that determines the next token. However, providers often support caching the results of those computations, such that if an identical prefix of a prompt has already been processed, the provider can reuse intermediate computations from the cache rather than recomputing them from scratch. That reduces the cost of processing those tokens, and that savings can be passed along to the consumer. As such, input tokens are often advertised with multiple rates, including a rate for input tokens that are read from the cache.LLM 提供商通常对输入 token(你发送给它们的内容)收取一个费率,对输出 token(它们发送给你的内容)收取另一个费率。计费通常按 token 进行,因为它们是执行推理所需的计算工作的有用近似:对于每个输入 token,模型必须读取它,将其纳入其内部表示,并将其用作确定下一个 token 的计算的一部分。然而,提供商通常支持缓存这些计算的结果,这样如果提示的一个相同前缀已经被处理过,提供商就可以重用缓存中的中间计算,而不是从头开始重新计算。这降低了处理这些 token 的成本,并且这些节省可以传递给消费者。因此,输入 token 通常以多种费率进行宣传,包括从缓存中读取的输入 token 的费率。

The discounts are steep! Often providers bill cache hits at a 90% discount, so for example a provider might charge $2.00 for 1 million input tokens but only $0.20 for 1 million cached input read tokens. In other words, you really, really want to maintain good prompt caching so that your bill is an order of magnitude smaller.折扣幅度很大!提供商通常以 90% 的折扣对缓存命中进行计费,例如,提供商可能会对 100 万个输入 token 收费 2.00 美元,但对 100 万个缓存输入读取 token 仅收费 0.20 美元。换句话说,你真的非常希望保持良好的提示缓存,以便你的账单能小一个数量级。

The data from the porting effort shows we did well here. The prompt-cache hit rate was 96.22%: cache reads divided by all input-side token volume (cache reads plus cache writes plus fresh input). Cache writes were 3.07%, and fresh input was 0.71%. This is not an accident. GitHub Copilot shapes the agent loop specifically to preserve a long and stable prefix (the system prompt, then the tool definitions, then the accumulated conversation), so each turn appends to context the model has already processed. The expensive part of the context is paid for once and then re-read at an order of magnitude less monetary cost on every subsequent call. It’s also the reason the economics of long autonomous sessions hold together at all. A three-hundred-hour port that re-read its entire growing context from scratch on each of tens of thousands of calls would cost a different order of magnitude than what we saw. Agent harness developers spend a great deal of energy trying to avoid breaking the prompt cache, and model vendors routinely ship new features to help them do so.移植工作的数据表明我们在这方面做得很好。提示缓存命中率为 96.22%:缓存读取除以所有输入侧 token 量(缓存读取 + 缓存写入 + 新鲜输入)。缓存写入为 3.07%,新鲜输入为 0.71%。这不是偶然的。GitHub Copilot 专门塑造了代理循环以保留一个长且稳定的前缀(系统提示,然后是工具定义,然后是累积的对话),因此每一轮都会追加到模型已经处理过的上下文中。上下文的昂贵部分支付一次,然后在随后的每次调用中以低一个数量级的货币成本重新读取。这也是长自主会话的经济性能够成立的原因。一个 300 小时的移植,如果要在数万次调用中的每一次都从头开始重新读取其整个增长的上下文,其成本将与我们所看到的完全不同。代理工具开发人员花费大量精力试图避免破坏提示缓存,模型供应商也定期发布新功能来帮助他们这样做。

Prompt-cache composition across the porting sessions: 96.22% cache reads, 3.07% cache writes, and 0.71% fresh input.

Compaction tells a complementary story. Across the port sessions, GitHub Copilot compacted context automatically 5,116 times (the moments when a session had filled its context window and summarized itself in order to keep going). The single sessions-infrastructure port pull request compacted 647 times over its many-day lifespan, while one small port never compacted once. Sustained multi-hundred-hour autonomous work is only possible because the agent can recycle its working memory over and over without losing the thread. Every one of those thousands of summarizations was a point where a lossy handoff could have quietly derailed the port, and mostly didn’t. Copilot’s subagents also greatly factor into minimizing compaction. Each subagent gets its own context, so a parent session can effectively ask a question, have a subagent go off and exert a fair amount of context in computing the answer, and then report just the answer back to the parent. The parent’s context needn’t be impacted by all of that intermediate information.压缩(Compaction)讲述了一个互补的故事。在移植会话中,GitHub Copilot 自动压缩上下文 5,116 次(当会话填满其上下文窗口并总结自身以继续进行时的时刻)。单个会话基础设施移植拉取请求在其多天的生命周期中压缩了 647 次,而一个小移植从未压缩过一次。持续的数百小时自主工作之所以可能,是因为代理可以反复回收其工作记忆而不会丢失线程。那数千次总结中的每一次都是一个可能发生有损移交并悄悄破坏移植的点,但大多数情况下并没有。Copilot 的子代理也在最大限度地减少压缩方面发挥了很大作用。每个子代理都有自己的上下文,因此父会话可以有效地提出一个问题,让子代理走开并投入相当多的上下文来计算答案,然后将答案报告回父会话。父会话的上下文不必受到所有这些中间信息的影响。

The “mostly didn’t” above is visible in the session logs. I had Copilot pair each successful compaction with the work surrounding it when at least 20 tool calls existed on both sides. That produced ~4,000 comparable windows. The mix of what the agent did in the 20 tool calls before compacting looks similar in scale to what it did after compacting (exploration 46.5% before and 48.1% after, mutation 8.4% before and 6.0% after, validation 4.7% before and 4.0% after, failures 1.0% before and 1.5% after). If compaction were regularly dropping the thread of thought, we’d expect the after side to be visibly re-orientation-heavy, with a spike in reading and a collapse in editing while the agent rediscovered where it was and what it should be doing. Instead, there’s only a mild shift in that direction.上面的“大多数情况下并没有”在会话日志中可见。我让 Copilot 将每次成功的压缩与当两侧至少存在 20 次工具调用时围绕它的工作配对。这产生了约 4,000 个可比较的窗口。代理在压缩前 20 次工具调用中所做的事情在规模上看起来与压缩后所做的事情相似(探索前 46.5%,后 48.1%;变异前 8.4%,后 6.0%;验证前 4.7%,后 4.0%;失败前 1.0%,后 1.5%)。如果压缩经常丢失思维线程,我们会期望后侧明显偏向于重新定位,伴随着阅读量的激增和编辑量的崩溃,同时代理重新发现它在哪里以及它应该做什么。相反,只有轻微的转向。

Yes, static analysis helps是的,静态分析有帮助

There’s a popular meme that Rust is an unusually good target for AI-generated code because Rust’s strict compiler catches what the model gets wrong. The session logs let us test that theory, at least for tasks that look like this porting effort.有一个流行的模因(meme)认为 Rust 是 AI 生成代码的异常好的目标,因为 Rust 严格的编译器会捕捉到模型出错的地方。会话日志让我们能够测试该理论,至少对于像这次移植工作这样的任务是这样。

Direct validation-command results captured 8,678 occurrences of rustc‘s error codes. The four largest diagnostic families cover 84%:直接验证命令结果捕获了 8,678 次 rustc 的错误代码。最大的四个诊断家族覆盖了 84%:

  • 37%: Name and import resolution, dominated by E0425 (“cannot find value in this scope”)37%:名称和导入解析,以 E0425(“在此范围内找不到值”)为主
  • 22%: Missing methods or fields22%:缺少方法或字段
  • 14%: Type mismatches14%:类型不匹配
  • 11%: Unsatisfied trait bounds11%:不满足 trait 约束

Every one of those is ordinary wiring: a name output slightly wrong, a signature that didn’t line up, a field that had been renamed, an abstraction left unimplemented. These are the kinds of mistakes bulk translation easily and accidentally produces and exactly the ones a compiler catches very quickly.每一个都是普通的布线:名称输出稍微错误,签名没有对齐,字段被重命名,抽象未实现。这些是批量翻译容易且偶然产生的错误,也正是编译器非常快速地捕捉到的错误。

But note what’s absent from that list: anything truly specific to Rust. Every one of those four categories is bread-and-butter static typing, and a C# or Java or Go compiler would catch all of them just as well, several of them with friendlier diagnostics, and all of them a great deal faster. If this is the argument for pointing agents at Rust, it’s really an argument for pointing them at any statically typed language. A strongly typed compiler and/or a language with excellent static analysis and linting is genuinely a good fit for this work, with the agents using it as a fast feedback loop. Across the 4,478 direct cargo check runs for which the stricter result matcher captured an outcome, 87.1% came back clean, which is what you get from editing in small increments and recompiling constantly.但请注意列表中缺少什么:任何真正特定于 Rust 的东西。这四个类别中的每一个都是静态类型的基本功,C#、Java 或 Go 编译器也能同样好地捕捉到所有这些,其中一些具有更友好的诊断,而且所有这些都快得多。如果这就是将代理指向 Rust 的论点,那么它实际上是将它们指向任何静态类型语言的论点。强类型编译器和/或具有出色静态分析和 linting 的语言确实非常适合这项工作,代理将其用作快速反馈循环。在 4,478 次严格结果匹配器捕获结果的直接 cargo check 运行中,87.1% 返回干净,这就是通过小增量编辑和不断重新编译所得到的结果。

In contrast, ownership, borrowing, and lifetime errors combined were only 1.7% of coded diagnostics. The borrow checker, the thing that dominates every conversation about Rust being hard, was a quiet background presence. The compiler spent almost all of its erroring energy on boring mechanical mistakes.相比之下,所有权、借用和生命周期错误加起来仅占编码诊断的 1.7%。借用检查器(那个主导了关于 Rust 很难的每一次对话的东西)是一个安静的背景存在。编译器几乎将其所有的错误精力都花在了无聊的机械错误上。

Agents like reading代理喜欢阅读

We can also examine the corpus of session events for tool call data, and from that extract some interesting observations about how agents spend their time.我们还可以检查会话事件语料库中的工具调用数据,并从中提取出一些关于智能体如何分配时间的有趣观察结果。

ToolCallsMedianMeasured hours
powershell630,4233 s2,833.9
view590,9880 s621.7
rg281,7831 s408.4
grep126,4831 s115.3
apply_patch53,7150 s17.0
edit40,5911 s24.1
read_powershell36,72890 s1,203.9
task13,080274 s2,329.0

My first takeaway here is that the agents spent far more time gathering evidence than changing code. Across the displayed file-reading and search tools versus the editing tools, they did 10x as much exploration as mutation. Reading files, searching the repository, and running diagnostic commands dominated; edits were a comparatively small effort. The popular image of AI spewing code is almost backwards; at this scale, the work looked much more like iterative investigation, inspecting the current state, forming a hypothesis, making a targeted change, rinsing and repeating.我的第一个发现是,智能体花费在收集证据上的时间远多于修改代码的时间。在展示的文件读取和搜索工具与编辑工具的对比中,它们进行探索的次数是进行修改次数的 10 倍。读取文件、搜索存储库和运行诊断命令占据了主导地位;编辑工作所付出的努力相对较小。AI 大量输出代码的普遍印象几乎是本末倒置的;在这种规模下,工作看起来更像是反复的调查:检查当前状态、形成假设、进行针对性修改,然后循环往复。

Delegation amplified that pattern. Subagents were used primarily to fan out exploration across independent questions, while the main agent was more likely to own the edits and integrate the answers. That’s a useful division of labor for this kind of project: many contexts can investigate in parallel, but keeping mutation closer to the coordinating agent reduces conflicting changes and preserves a coherent implementation strategy.任务委派放大了这种模式。子智能体主要用于将探索任务分发到各个独立的问题上,而主智能体则更有可能负责代码编辑和答案整合。对于这类项目,这是一种实用的分工方式:多个上下文可以并行进行调查,而将修改工作保持在协调智能体附近,可以减少冲突的变更并保持实施策略的一致性。

The shell traffic also shows how much of autonomous software work is state management. Read-only Git inspection was the most common command pattern because the agents were constantly asking, effectively, “where am I?” They were looking for what had changed, what a rebase had done, what another session had landed, and how far a branch had drifted from a rapidly evolving main. That orientation work enabled many long-running efforts to operate against the same moving codebase without blindly overwriting one another.Shell 流量也显示了自主软件开发中有多少工作属于状态管理。只读的 Git 检查是最常见的命令模式,因为智能体一直在不断地确认“我在哪里?”。它们在寻找发生了什么变化、变基(rebase)做了什么、另一个会话提交了什么,以及分支与快速演进的 main 分支偏离了多远。这种定位工作使得许多长期运行的任务能够在同一个不断变动的代码库上进行操作,而不会盲目地覆盖彼此。

Looking inside the shell-tool traffic, the most common command families make that balance between orientation and validation even clearer:深入观察 Shell 工具的流量,最常见的命令族让定位与验证之间的平衡变得更加清晰:

Command familyCallsMedianMeasured hours
git inspect300,5302 s608.1
git other89,8653 s243.1
search85,4822 s147.6
pnpm test13,85222 s219.1
pnpm lint9,75729 s177.0
cargo test8,437120 s364.2
git commit7,41011 s39.7
cargo fmt5,22318 s77.2
cargo check4,492120 s176.9
pnpm build3,630180 s215.6
cargo clippy2,115135 s107.4
git rebase2,4967 s9.9
cargo build566104 s20.3

Model selection模型选择

GitHub Copilot lets a single session change models mid-conversation and lets different sessions run different models, so model choice became a per-slice decision. Two different kinds of model decision show up in the logs. On the main thread, the one driving each port, we chose the model and the reasoning effort. Inside a session, when the agent spun up subagents or subsessions to explore or to grind through a bounded task, the orchestrating model chose those models.GitHub Copilot 允许单个会话在对话中途更换模型,也允许不同的会话运行不同的模型,因此模型选择成为了针对每个切片的决策。日志中出现了两种不同类型的模型决策。在驱动每个端口的主线程上,我们选择了模型和推理工作量。在会话内部,当智能体启动子智能体或子会话来探索或处理受限任务时,编排模型会选择这些子模型。

Weekly model mix across primary porting sessions, concentrated in a small number of models and shifting over the course of the port.

For subagents, the model mix looks a bit different, with an agent rather than a human optimizing for throughput and cost rather than for the hardest judgment calls. The subagents it spawned most often ran on Claude Opus 4.8, GPT-5.6 Sol, Claude Haiku 4.5, and GPT-5.5, followed by Gemini 3.1 Pro and Claude Opus 5. However, at least at the time the ports were happening, three frequently used agent definitions pinned their model choice (explore and task to Claude Haiku, research to Claude Sonnet), so a significant part of that volume was determined by the choice of subagent rather than by choosing a model separately.对于子智能体,模型组合看起来有所不同,因为是由智能体而非人类进行优化,侧重于吞吐量和成本,而非最艰难的判断。它最常启动的子智能体运行在 Claude Opus 4.8、GPT-5.6 Sol、Claude Haiku 4.5 和 GPT-5.5 上,其次是 Gemini 3.1 Pro 和 Claude Opus 5。然而,至少在进行端口迁移时,三个常用的智能体定义固定了它们的模型选择(explore 和 task 使用 Claude Haiku,research 使用 Claude Sonnet),因此很大一部分工作量是由子智能体的选择决定的,而不是单独选择模型。

Weekly model mix across subagents spawned during the port, with volume concentrated in Claude Opus 4.8, GPT-5.6 Sol, Claude Haiku 4.5, and GPT-5.5.

Working with agent fleets与智能体集群协作

The GitHub Copilot app’s support for visualizing active pull request sessions, the status of each, and easily switching between them made it ideal for managing lots of concurrent work inherent to the port. But one of the things that really made it shine was its ability for sessions to interact with other sessions.GitHub Copilot 应用支持可视化活跃的拉取请求(PR)会话及其状态,并能轻松在它们之间切换,这使其非常适合管理端口迁移中固有的许多并发工作。但真正让它脱颖而出的是会话之间相互交互的能力。

A session can create other sessions, and it can message other sessions while they run. Each session, parent or child, gets its own worktree, its own branch, and its own agent loop; it’s separate from the session that spawned it rather than something running inside it. That’s different from a subagent, which runs inside the parent’s own workspace and hands its answer back into the parent’s context. Both are useful constructs for different things.一个会话可以创建其他会话,并且可以在运行过程中向其他会话发送消息。每个会话(无论是父会话还是子会话)都有自己的工作树、自己的分支和自己的智能体循环;它与生成它的会话是分离的,而不是在其中运行的某种东西。这与在父会话工作区内运行并将答案返回到父会话上下文中的子智能体不同。两者都是针对不同场景的有用结构。

As an example of how a session might create other sessions, one of the hardest ports was for the session.ts file. This file had grown organically to be ~30,000 lines of TypeScript. It represented the backbone of a session and effectively spanned horizontally across the whole runtime, touching and being touched by practically every component, sitting at the center of state, events, tools, models, hooks, persistence, and entrypoint access. As a result, I left it for close to the end of the porting process, working up from the bottom of the stack across all the verticals until they all dead ended at session.ts. The porting session that took it on did not start by just diving in and writing Rust. It spent its first fifty-six minutes reading, with 122 tool calls before it created anything, building a picture of what the file actually owned and where the seams were. Only then did it start delegating, logically splitting up the file and delegating slices to subsessions. Across the whole 25-hour run, it made 222 shell calls, 205 file views, and 197 ripgrep searches of its own, on top of everything its child sessions did.以一个会话如何创建其他会话为例,最困难的迁移之一是 session.ts 文件。这个文件已经自然增长到约 30,000 行 TypeScript 代码。它代表了会话的骨干,实际上横跨了整个运行时,触及并被几乎所有组件触及,处于状态、事件、工具、模型、钩子、持久化和入口访问的核心。因此,我将它留到迁移过程的最后,从栈底向上遍历所有垂直领域,直到它们都在 session.ts 处汇合。负责此任务的迁移会话并没有直接开始编写 Rust 代码。它花了前 56 分钟进行阅读,在创建任何内容之前进行了 122 次工具调用,构建了一幅关于该文件实际拥有什么以及边界在哪里的蓝图。只有在那之后,它才开始委派任务,在逻辑上将文件拆分并将切片委派给子会话。在整个 25 小时的运行过程中,除了其子会话所做的一切之外,它还进行了 222 次 shell 调用、205 次文件查看和 197 次 ripgrep 搜索。

Fifteen child sessions nested under the session that spawned them.

That’s 15 child sessions, each one a separate branch with its own worktree and a separate agent, all created implicitly by the parent session at the top. The parent session created them in seven waves over about three hours: the first wave created five, the second another two about twenty minutes later, then another pair twenty minutes after that, then singles and pairs spread out over the next two hours.那是 15 个子会话,每一个都是一个单独的分支,拥有自己的工作树和独立的智能体,所有这些都是由顶层的父会话隐式创建的。父会话在大约三个小时内分七波创建了它们:第一波创建了五个,大约二十分钟后第二波创建了另外两个,然后二十分钟后又是一对,接下来的两个小时里又零星地创建了单个或成对的会话。

Model choice was made per slice: 10 of the 15 ran on GPT-5.6 Sol and five on Claude Opus 4.8. All 15 were started in GitHub Copilot’s autopilot mode, which lets a session pursue an objective without stopping for approval at each step. The median kickoff prompt was about 1,100 characters, long enough to carry the ownership boundary and the constraints, but short enough that the child session had to work out the approach itself. I prompted the parent agent, and then the parent agent, not a human, wrote those kickoff prompts to each child session.模型选择是按切片进行的:15 个中有 10 个运行在 GPT-5.6 Sol 上,5 个运行在 Claude Opus 4.8 上。所有 15 个会话都是在 GitHub Copilot 的自动驾驶模式下启动的,该模式允许会话在不因每一步都等待批准而停止的情况下追求目标。中位数的启动提示词大约为 1,100 个字符,足够长以传达所有权边界和约束,但又足够短,使得子会话必须自己找出实现方法。我提示了父智能体,然后是父智能体(而非人类)向每个子会话编写了这些启动提示词。

Alongside those 15 subsessions, the same parent session also employed five subagents: three explore agents fired in parallel with the first wave, one code-review, and one rubber-duck. These subagents explored questions, feeding back to the parent answers it needed in its context before it could decide what to do next. The subagents allowed the parent to get deeply thought out answers without needing to spend its own context window on deriving them.除了那 15 个子会话外,同一个父会话还雇用了 5 个子智能体:3 个 explore 智能体与第一波会话并行触发,1 个 code-review 智能体和 1 个 rubber-duck 智能体。这些子智能体探索问题,将父智能体在决定下一步行动前所需的答案反馈给它。子智能体允许父智能体获得经过深思熟虑的答案,而无需消耗其自身的上下文窗口来推导它们。

In contrast, child sessions went to work on the actual porting, the work that produced a diff and needed isolation from the other parallel porters. The child sessions’ work touched 140 distinct files in the repo, 120 of which were touched by exactly one session. The 20 contended files were all hubs, such as session.ts itself. But each session was working in its own worktree, and so was able to proceed undisturbed by its siblings. The parent, of course, paid for that in coordination. It spent a good deal of effort communicating with its child sessions, acting as an information broker, polling their state 60 times and sending 89 coordination messages. When the child sessions each announced their completion, the parent cherry-picked their commits into its own branch and resolved the conflicts. These were not particularly clean merges, either, and the parent agent spent a decent amount of time reconciling the edits.相比之下,子会话投入到实际的迁移工作中,这些工作产生了差异(diff)并需要与并行进行的其他迁移任务隔离。子会话的工作触及了存储库中的 140 个不同文件,其中 120 个文件恰好由一个会话触及。那 20 个有争议的文件都是枢纽,例如 session.ts 本身。但每个会话都在自己的工作树中工作,因此能够不受其兄弟会话的干扰继续进行。当然,父会话为此付出了协调代价。它花费了大量精力与子会话沟通,充当信息代理,轮询它们的状态 60 次并发送了 89 条协调消息。当子会话各自宣布完成时,父会话将它们的提交拣选(cherry-pick)到自己的分支中并解决冲突。这些合并并不特别干净,父智能体花费了相当多的时间来协调这些编辑内容。

We can see that on a timeline, which shows the parent session and most of its child sessions.我们可以在时间轴上看到这一点,它显示了父会话及其大多数子会话。

Timeline of the session.ts port showing one 25-hour parent session using five subagents and spawning 15 child sessions in seven waves.

Note those large gaps. I was traveling while working on this port, and I had to close my laptop at various points. (I subsequently changed my workflow to incorporate cloud-based virtual machines I could remote into.)注意那些巨大的空白。我在进行此项迁移时正在出差,不得不在不同时间合上笔记本电脑。(后来我调整了工作流程,引入了可以远程连接的云端虚拟机。)

These parallel child sessions had a significant impact on that laptop. For a while, the concurrent porting was going swimmingly. Then all 15 concurrent agents on one machine each tried to build and test, and my poor laptop ground to a halt. I prompted to the parent, asking it to relay to its child sessions that they must all stop building and testing. The parent relayed that constraint outward, and they thankfully killed their builds and proceeded to work with minimal CPU activity. I subsequently updated my standing instructions that subagents and subsessions should avoid large builds and test runs while porting, instead deferring that to be done only by the parent agent.这些并行的子会话对那台笔记本电脑产生了重大影响。有一段时间,并发迁移进行得非常顺利。然后,单台机器上的所有 15 个并发智能体都试图进行构建和测试,我可怜的笔记本电脑陷入了停滞。我向父会话发送提示,要求它向子会话传达它们必须停止构建和测试。父会话将该约束向外传递,它们幸运地终止了构建,并以最小的 CPU 活动继续工作。随后,我更新了我的长期指令,要求子智能体和子会话在迁移时应避免进行大型构建和测试,而是推迟到仅由父智能体执行。

Later I took that one step further and made an otherwise ordinary chat session into a build scheduler for eight independent porting sessions. The prompt was embarrassingly simple: send every open session a policy to avoid CPU-intensive building and testing where possible, require it to ask this session for permission when a build was necessary, and act as a gate, handing out the ability for one session at a time to build. Basically I turned the chat session into an agentic mutex. The gate kept an explicit owner and queue and granted one lease at a time through the same cross-session messaging mechanism the sessions already used to coordinate code. Sessions that asked for and were denied the lease often waited by doing other work in the meantime, like picking off things from its todo list.后来,我更进一步,将一个原本普通的聊天会话变成了一个负责八个独立迁移会话的构建调度器。提示词简单得令人尴尬:向每个打开的会话发送一项策略,尽可能避免 CPU 密集型的构建和测试,要求它们在必要时向此会话请求许可,并充当门卫,一次只授予一个会话构建的能力。基本上,我把聊天会话变成了一个智能互斥锁(agentic mutex)。门卫保留了一个明确的所有者和队列,并通过会话间已用于协调代码的相同消息传递机制,一次授予一个租约。请求并被拒绝租约的会话通常会通过执行待办事项列表中的其他工作来等待。

One chat session acting as the build resource gate for eight porting sessions.

This session.ts port was also involved in one of the coolest, saddest, and certainly most unexpected interactions I witnessed during the whole runtime port. As I mentioned, we did the port primarily bottom-up, which is why session.ts that effectively sits on top of every other component was one of the last components ported. The only thing consistently above session.ts are all of the entrypoints into the runtime, namely the public functions that are exposed from the SDK and that show up in the previously discussed dispatch table. There are hundreds of these. And while I know that many of them immediately call into session.ts, I wanted to get a jump on the porting, and so after launching the session.ts session, I launched a session to port all of the entrypoints. I told it to stop at the session.ts boundary. I figured there may be a bit of throwaway work and some amount of effort or number of tokens needed in a rebase, but that it would accelerate the overall porting. Then I went to bed. And then… they found each other.这次 session.ts 迁移还涉及我在整个运行时迁移过程中见证的最酷、最悲伤、当然也是最出乎意料的交互之一。正如我提到的,我们主要是自下而上进行迁移的,这就是为什么 session.ts 这个实际上位于所有其他组件之上的组件是最后迁移的组件之一。唯一始终位于 session.ts 之上的是运行时中的所有入口点,即从 SDK 公开的公共函数,它们出现在前面讨论的调度表中。这些函数有数百个。虽然我知道它们中的许多会立即调用 session.ts,但我想加快迁移进度,因此在启动 session.ts 会话后,我启动了一个会话来迁移所有入口点。我告诉它在 session.ts 边界处停止。我想可能会有一些浪费的工作,以及在变基中需要一定的工作量或 token 数量,但它会加速整体迁移。然后我就去睡觉了。然后……它们发现了彼此。

My kickoff prompt for the entrypoints session did tell it that the session port and six component ports were running concurrently, as I wanted it to know its boundaries and what it should avoid porting to avoid as many conflicts as possible. Apparently my prompting had the opposite effect. Just over four minutes in, having inventoried the ingress paths and presumably formed a view of how much they overlapped, it invoked an app built-in orchestrate skill, whose purpose is coordinating work across sessions. From there:我对入口点会话的启动提示词确实告诉它,session 迁移和六个组件迁移正在并发运行,因为我希望它知道自己的边界以及应该避免迁移的内容,以尽可能减少冲突。显然,我的提示产生了相反的效果。仅仅四分钟多一点,在清点了入口路径并推测形成了它们重叠程度的视图后,它调用了一个内置的 orchestrate 技能,该技能的目的是协调跨会话的工作。从那里开始:

  1. The entrypoints session enumerated every active session and sent messages to the ones with perceived overlap.入口点会话枚举了每个活跃会话,并向那些被认为有重叠的会话发送了消息。
  2. The session.ts session answered with a 2,001 character inventory titled “Concrete overlap on stephentoub-port-session-to-rust“session.ts 会话回答了一个 2,001 字符的清单,标题为“关于 stephentoub-port-session-to-rust 的具体重叠”
  3. The entrypoints session read the session.ts session’s worktree to confirm what it had just been told (trust but verify, I guess).入口点会话读取了 session.ts 会话的工作树以确认它刚刚被告知的内容(我想这就是“信任但核实”)。
  4. The entrypoints session asked the session.ts session whether it was ready to reconcile its 760 file diff.入口点会话询问 session.ts 会话是否准备好协调其 760 个文件的差异。
  5. The session.ts session basically told it to get lost: “Not ready to commit/integrate.”session.ts 会话基本上让它走开:“尚未准备好提交/集成。”
  6. The entrypoints session proceeded to ask the same question three more times, and each time it got back the same answer from the session.ts session.入口点会话继续又问了三次同样的问题,每次都从 session.ts 会话那里得到了同样的回答。
  7. At which point the entrypoints session decided it didn’t care what the session.ts session thought and simply reached into its worktree and grabbed all of the other session’s changes and merged them into its own.此时,入口点会话决定它不在乎 session.ts 会话的想法,直接进入其工作树,抓取了另一个会话的所有更改并将其合并到自己的分支中。
  8. Then both sessions went on their merry way.然后两个会话都各自继续前行了。

A few things I took from this interaction:我从这次交互中总结了几点:

  1. It’s important to be explicit about intent. The kickoff prompt named the other running sessions so this one would know what to leave alone. But I didn’t make that “leave alone” part explicit, so instead of blocking the agent from doing something, I ended up encouraging it to do it. I needed to be much more explicit in my intent and guidance.明确意图非常重要。启动提示词命名了其他正在运行的会话,以便这个会话知道哪些内容不要动。但我没有明确指出“不要动”的部分,所以与其说是阻止智能体做某事,我最终反而鼓励它去做了。我需要更明确地表达我的意图和指导。
  2. Whatever you make available is something an agent may decide applies. The orchestrate skill ships in the GitHub Copilot app and describes itself as being for running independent workstreams in parallel. Nothing in my prompt mentioned it. The model discovered its own situation, matched it against that description, and loaded it. The set of capabilities you expose is the set of behaviors you might get, including in situations you never pictured.你提供的任何功能,智能体都可能认为适用。orchestrate 技能随 GitHub Copilot 应用发布,并描述为用于并行运行独立工作流。我的提示词中没有提到它。模型发现了自己的处境,将其与该描述匹配,并加载了它。你暴露的能力集就是你可能得到的行为集,包括在你从未设想过的情况中。
  3. Peers need a tiebreaker. Neither session could compel the other. When the session.ts session said it was not ready to integrate four separate times, that refusal carried no weight, so the session willing to act unilaterally won by default. Parallel sessions over adjacent code need a designated coordinator, or they need a human, and these had neither.同行之间需要仲裁者。两个会话都无法强迫对方。当 session.ts 会话四次表示未准备好集成时,该拒绝没有任何分量,因此愿意单方面行动的会话默认获胜。相邻代码上的并行会话需要指定的协调者,或者需要人类,而这些会话两者都没有。
  4. “Run autonomously” needs an exception for decisions that reach outside your own branch. I really meant “don’t wake me up over design details.” It heard (not unreasonably) that annexing a peer was in scope. Again, I should have been more explicit in my guidance.“自主运行”需要针对超出你自身分支的决策设置例外。我确实是指“不要因为设计细节而叫醒我”。它(并非不合理地)听成吞并同行在范围内。同样,我应该在指导中更明确一些。
  5. The root cause here was me. I partitioned this work top-down and bottom-up at the same time, and the two directions met in the middle at the single most connected file in the codebase. I was too greedy to make forward progress. Everything above follows from that.根本原因是由于我。我同时从上到下和从下到上划分了这项工作,这两个方向在代码库中连接最紧密的文件处相遇。我太贪心于取得进展了。以上所有内容都源于此。

This whole interaction was, thankfully, an interesting outlier. Across the whole runtime porting effort, most of the leaf component ports were straightforward single-session tasks. The larger subsystems often involved multiple subsessions and subagents. How those pieces participated over the course of a port varied significantly, though.谢天谢地,整个交互是一个有趣的异常值。在整个运行时迁移工作中,大多数叶子组件迁移都是直接的单会话任务。较大的子系统通常涉及多个子会话和子智能体。然而,这些部分在迁移过程中如何参与差异很大。

The port of model orchestration, the layer that actually talks to the providers, provides a good example of one pattern. Its main session ran 42 wall-clock hours and started 126 subagents. At its busiest, 22 were working at the same time. However, the majority of the time, it was only the main agent, and then now and again it would spawn a significant number of subagents for a window of time.模型编排的迁移(与提供商实际对话的层)提供了一个模式的好例子。其主会话运行了 42 个挂钟小时并启动了 126 个子智能体。最忙时,有 22 个在同时工作。然而,大部分时间里,只有主智能体在工作,偶尔它会在一段时间内生成大量子智能体。

The 42-hour model orchestration port generated most code in its first 12 hours, followed by a distinct validation and review phase involving 126 subagents.

Three things in that picture stand out. First, effectively all of the code generation was done in the first 12 hours; the next day of work after that is all validation. Second, the color of the bottom row shifts left to right, from predominantly blue and green (reading, building) to predominantly blue and orange (reading, reviewing); that makes logical sense, but it’s neat to see it in practice. Third, that example, and more generally this pattern, has very clean separation between phases. The extension-runtime port is a counter-example.图中的三点很突出。首先,实际上所有的代码生成都在前 12 小时内完成;之后的一天工作全是验证。其次,底行的颜色从左到右发生变化,从主要是蓝色和绿色(读取、构建)变为主要是蓝色和橙色(读取、审查);这在逻辑上是有道理的,但在实践中看到它很有趣。第三,该示例,以及更广泛地说这种模式,在阶段之间有非常清晰的分离。扩展运行时迁移是一个反例。

The 88-hour extension runtime port interleaved reading, writing, building, and reviewing throughout most of the session rather than separating them into phases.

It took 88 hours instead of 42 and had very different structure:它花费了 88 小时而不是 42 小时,并且结构截然不同:

  • Writing and reviewing overlap significantly. Whereas in the previous example, the work was very waterfall (first code generation, then review), here reviewing starts long before writing stops, and both proceed and overlap for most of the duration of the effort.编写和审查有显著重叠。在前面的例子中,工作非常瀑布式(先代码生成,后审查),而在这里,审查在编写结束前很久就开始了,并且两者在大部分工作期间都在进行和重叠。
  • The bottom row is all over the place color-wise. Where model orchestration shifts from green to orange as it moves from writing to checking, this one is the same mixture of reading, building, and reviewing from beginning to end. The middle half of the reading calls is spread across a 49-hour span, the writing across 33 hours, and the reviewing across 27, inside an 88-hour session. Every category is spread across most of the run.底行在颜色上到处都是。模型编排从编写转向检查时颜色从绿色变为橙色,而这个从头到尾都是读取、构建和审查的混合。读取调用的中间一半分布在 49 小时的跨度内,编写分布在 33 小时内,审查分布在 27 小时内,都在 88 小时的会话中。每个类别都分布在大部分运行时间内。
  • Idling is deferred to the end. The fleet works nearly continuously for the first 56 hours.闲置被推迟到最后。集群在前 56 小时内几乎连续工作。
  • The proportions still match. Writing Rust is 2% of tool calls here against 1% there, reading 44% against 57%, reviewing 23% against 27%. The two sessions agree on what the work to be done is, just not on when it happens.比例仍然匹配。编写 Rust 的工具调用占比为 2%,而另一个是 1%;读取占比 44% 对 57%;审查占比 23% 对 27%。这两个会话对于要完成的工作是一致的,只是在何时发生上不一致。

Those blank slices at the end are also a visual representation of a problem becoming more and more common in this agentic coding era: waiting for approval. Someone on the team and/or an agent reviews the code and leaves feedback, there’s a brief period of activity where the agent addresses the feedback and drives CI to green again, and then more waiting, rinse and repeat, until eventually we get the dopamine-inducing stamp of approval.最后那些空白切片也是在这个智能体编码时代变得越来越普遍的一个问题的视觉表现:等待批准。团队中的某人或智能体审查代码并留下反馈,有一段短暂的活动期,智能体处理反馈并再次推动 CI 变绿,然后是更多的等待,循环往复,直到最终我们得到令人多巴胺分泌的批准印章。

These two examples each represent a dominant pattern. About a fourth of the sessions look more like the model orchestration session, while three quarters look like the extensions runtime one. The clean progression through phases was the exception; the common case was the agent planning, writing, and reviewing all the way through.这两个例子各代表一种主导模式。大约四分之一的会话看起来更像模型编排会话,而四分之三看起来像扩展运行时会话。阶段间的清晰推进是个例外;常见情况是智能体在整个过程中进行规划、编写和审查。

Code review at scale大规模代码审查

Much of that focus on review visible in the previous graphs was due to my explicit prompting. I created a simple prompt-as-a-skill I called rust-rebase-review (in addition to a general Rust coding skill we have merged in the repo). Due to the fast rate of incoming changes, many of which conflicted, I was frequently rebasing. Via custom instructions, I encouraged the harness to invoke this skill at appropriate points in the process, and also manually invoked it from time to time. The prompt evolved a bit over time, but it was a variation of this:前面图表中可见的对审查的关注很大程度上归功于我的明确提示。我创建了一个简单的“提示词即技能”,我称之为 rust-rebase-review(除了我们在存储库中合并的通用 Rust 编码技能外)。由于传入更改的速度很快,其中许多存在冲突,我经常进行变基。通过自定义指令,我鼓励工具在流程的适当点调用此技能,并时不时手动调用它。提示词随着时间推移有所演变,但它是以下内容的变体:

Squash into a single commit, then rebase on the latest in origin/main, resolving all conflicts, and force push. As part of rebasing, pay extra special attention to anything that has changed, been added, been removed, and ensure that logic is all ported over to the corresponding Rust code correctly. Always do the rebasing yourself / in the main agent; do not spawn a subagent for it.

Then enter a review/fix loop where you launch a subagent per opus 5, gpt-5.6-sol, and grok 4.6.

- That subagent should do a line-by-line comparison of the old TypeScript and the new Rust, confirming behavioral equality.
- Look for anything introducing any kind of incompatibility; our goal is to move this code into Rust with as close as is possible to 100% the same semantics. If you hit anything questionable, ask me about it.
- We want to ensure we're writing as efficient and idiomatic Rust code as we can; look for opportunities to simplify, to use routines like from the memchr crate to optimize searches instead of open-coded loops, avoid unnecessary allocation, use traits for reuse and loose coupling, etc.
- Ensure that all defunct TypeScript code (e.g. code that has been fully ported, tests that are now no longer necessary because they're duplicative, unnecessary napi shims, etc.) has been deleted.
- Ensure that we've ported as much code as possible, e.g. if there's any TypeScript remaining in touched files and that TypeScript is more than just a shim, that's a red flag. If new TypeScript that's not just a super thin shim is being added, that's a red flag. Look for any callers of TypeScript shims to see whether those callers can instead be ported to Rust, pushing the boundary as far as reasonably possible. Our goal is to soon get to 100% Rust in the runtime layer.
- Validate that no E2E tests have been deleted or changed. Such changes are an indication of a porting bug.

If a review surfaces issues, validate them, and then if there are any to fix, fix them, and iterate to do another full review. Continue iterating with reviewing/fixing until all reviews come back clean. After every set of changes in response to review feedback, commit and push so that CI validation runs concurrently with subsequent reviews.

Don't bother running full test suites; that'll be handled in CI. Try to minimize CPU-consuming efforts to the bare minimum, as we'll likely have many operations happening concurrently.

With the frequent rebasing, it’s easy for incoming changes to get lost accidentally. But we found an unexpected benefit to the in-place atomic swap: by deleting the TypeScript at the same time as we were adding the corresponding Rust, we were implicitly creating conflicts with rebase-induced incoming changes to that TypeScript: one branch changing it and the other deleting it. This guaranteed we’d notice changes to already ported code, rather than needing to rationalize for every incoming line whether it was something that might have been touching previously ported code.由于频繁的变基,传入的更改很容易意外丢失。但我们发现原地原子交换有一个意想不到的好处:通过在添加相应 Rust 代码的同时删除 TypeScript 代码,我们隐式地与 rebase 引入的对该 TypeScript 的传入更改产生了冲突:一个分支在修改它,另一个分支在删除它。这保证了我们会注意到对已迁移代码的更改,而不是需要为每一行传入的代码合理化它是否可能触及了先前迁移的代码。

My own reviews were of course only a portion of the agentic review being performed. In addition to CCR running on every commit, the team has multiple dedicated code review bots, each with their own approaches and prompts, running on every commit and providing detailed feedback. All of this would end up in comments on the pull requests that would then need to be addressed. Thankfully, handling all of that agentic feedback can also be handled agentically (mostly).我自己的审查当然只是所执行的智能体审查的一部分。除了在每次提交上运行的 CCR 外,团队还有多个专门的代码审查机器人,每个都有自己的方法和提示词,在每次提交上运行并提供详细反馈。所有这些最终都会出现在拉取请求的评论中,然后需要进行处理。谢天谢地,处理所有这些智能体反馈也可以(大部分)通过智能体方式处理。

For my review, I focused on architecture, design, conventions, and approach. The agents did the exhaustive old-versus-new comparisons; tests and static analysis checked mechanically enforceable properties; human reviewers concentrated on architecture, API contracts, risk, and any suspicious places surfaced by those other layers.对于我的审查,我专注于架构、设计、约定和方法。智能体进行了详尽的新旧对比;测试和静态分析检查了机械可强制执行的属性;人类审查员专注于架构、API 合约、风险以及由这些其他层浮现出的任何可疑之处。

I chose the destination architecture, decided what behavior mattered, partitioned the work, resolved ambiguous trade-offs, judged the evidence, manually reviewed high-risk areas, reviewed agentic responses to feedback, and made the final merge decisions. The agents changed the amount of code one engineer could supervise. They did not remove the need for an engineer who understood the system and could vouch for the direction, the guardrails, and the release.我选择了目标架构,决定了哪些行为重要,划分了工作,解决了模糊的权衡,判断了证据,手动审查了高风险区域,审查了智能体对反馈的响应,并做出了最终的合并决定。智能体改变了一名工程师可以监督的代码量。它们并没有消除对一名了解系统并能为方向、护栏和发布负责的工程师的需求。

Automating the inner loop自动化内部循环

The GitHub Copilot app was central to this work. It manages lots of concurrent active sessions, making it easy to switch between them and carry along with each all the relevant paired context (associated terminal windows, browser windows, canvases, and so on). The feature that mattered most here is agent merge:GitHub Copilot 应用是这项工作的核心。它管理着许多并发的活跃会话,使切换它们变得容易,并随之携带所有相关的配对上下文(关联的终端窗口、浏览器窗口、画布等)。在这里最重要的功能是智能体合并(agent merge):

The GitHub Copilot app's Agent Merge panel tracking review feedback, CI, conflicts, and merge readiness.

Agent merge is a loop built into the app (the CLI has it as well with /pr auto). On a timer or in response to external stimuli (like notifications from GitHub of CI completion or a review comment), the app will look to see what has changed. If a review comment has been left, it’ll invoke the agent to decide whether to reject the comment or to accept and address it (and respond, noting it’s automation that’s responding). If a test fails, it’ll download the logs, investigate the failure, and fix the bug. If a conflict occurs, it’ll invoke the agent to merge or rebase. Effectively, it automates the loop we as human developers all do, driving our pull requests to “green”, getting sign-off, and eventually merging.智能体合并是内置在应用中的一个循环(CLI 也通过 /pr auto 具备此功能)。在定时器上或响应外部刺激(如来自 GitHub 的 CI 完成通知或审查评论)时,应用会查看发生了什么变化。如果留下了审查评论,它会调用智能体来决定是拒绝评论还是接受并处理它(并回复,注明是自动化在响应)。如果测试失败,它会下载日志、调查失败原因并修复错误。如果发生冲突,它会调用智能体来合并或变基。实际上,它自动化了我们作为人类开发者都会做的循环,推动我们的拉取请求至“绿色”,获得签字,并最终合并。

Agent merge handled every single porting pull request. In most cases we stopped short of the actual “merge” part, however. The agent would fix all CI failures, address and respond to all comments, and make sure all conflicts were resolved. Before merging, I’d spot check what the agent actually did, and in particular how it addressed feedback. Did I disagree with any of its responses to reviewers? Was the high-level direction of applied fixes desirable and sound? For these ports, I typically left that last entry unchecked.智能体合并处理了每一个迁移拉取请求。然而,在大多数情况下,我们没有走到实际的“合并”部分。智能体会修复所有 CI 失败,处理并回复所有评论,并确保所有冲突都已解决。在合并之前,我会抽查智能体实际做了什么,特别是它如何处理反馈。我是否不同意它对审查员的任何回复?所应用修复的高层方向是否理想且合理?对于这些迁移,我通常将最后一步检查留空。

That last checkbox mattered more than once. In one merge loop pass, the port had deleted one of the functions exposed to the SDK. Our repo’s schema compatibility CI leg did exactly its job and failed. The agent’s response was to apply the repo’s schema-break-ok automation label, which is the escape hatch to make the check pass. In reviewing the pull request before merge, I asked the obvious question: “What is the schema break? You added the schema-break-ok label to the pull request; why is it ok?” It wasn’t. The method existed on main; the port had simply lost it. I called it an unacceptable regression and told the agent to bring it back fully in Rust. Twenty-one seconds later, the waiver was removed, and the method was restored with a native Rust implementation.最后一个复选框不止一次地发挥了作用。在一次合并循环中,迁移删除了 SDK 公开的函数之一。我们存储库的模式兼容性 CI 环节完全履行了其职责并失败了。智能体的响应是应用存储库的 schema-break-ok 自动化标签,这是使检查通过的逃生舱。在合并前审查拉取请求时,我问了一个显而易见的问题:“模式中断是什么?你给拉取请求添加了 schema-break-ok 标签;为什么它是可以的?”它并不可以。该方法存在于 main 分支上;迁移只是丢失了它。我称之为不可接受的回归,并告诉智能体在 Rust 中将其完全带回。21 秒后,豁免被移除,该方法通过原生 Rust 实现恢复了。

We responded to failures both locally and globally, fixing the individual cases but then also fixing the system so that they’d be less likely to recur. We continually evolved the instructions fed to the coding and review agents to further reduce the chances of these same issues happening again in future porting pull requests. We turned session logs into evals. And in some cases, we actually used the lessons learned to improve the runtime itself, via tweaks to prompts or tool descriptions or how autopilot operated.我们对本地和全局的失败都做出了响应,修复了个案,但也修复了系统,以便它们不太可能再次发生。我们不断改进提供给编码和审查智能体的指令,以进一步减少这些相同问题在未来迁移拉取请求中再次发生的可能性。我们将会话日志转化为评估。在某些情况下,我们实际上利用学到的经验教训改进了运行时本身,通过对提示词、工具描述或自动驾驶操作方式的调整。

Two migrations in one两次迁移合二为一

A language rewrite is almost never only a language rewrite. Every library the runtime leaned on had to be replaced, too, and unlike in the Rust code we wrote and owned, those replacements weren’t ours to make faithful. Some were the same idea under a different name. Some took several crates to cover what one npm package had done. A few had no acceptable off-the-shelf answer at all and had to be written by hand (by agent).语言重写几乎从来不仅仅是语言重写。运行时所依赖的每个库也都必须被替换,而且与我们编写和拥有的 Rust 代码不同,这些替换并非由我们来保证忠实度。有些是在不同名称下的相同想法。有些需要几个 crate 才能覆盖一个 npm 包所做的工作。少数根本没有可接受的现成答案,必须手动(由智能体)编写。

The CLI and runtime are currently in the same repo, sharing a package.json. Over the course of the port, we removed ~60 npm dependencies because they were only being used by runtime code that was ported to Rust. This is a lower bound for removals, because there were packages replaced for the runtime but that were still needed by the CLI. For example, zod is a TypeScript schema declaration and validation library that both the CLI and the runtime were using. With the Rust port, the runtime now uses a combination of serde, schemars, and jsonschema to satisfy the same purposes, yet zod is still in the manifest for the CLI’s sake.CLI 和运行时目前在同一个存储库中,共享一个 package.json。在迁移过程中,我们移除了约 60 个 npm 依赖项,因为它们仅被迁移到 Rust 的运行时代码使用。这是移除数量的下限,因为有些包是为运行时替换的,但 CLI 仍然需要它们。例如,zod 是一个 TypeScript 模式声明和验证库,CLI 和运行时都在使用它。通过 Rust 迁移,运行时现在使用 serde、schemars 和 jsonschema 的组合来满足相同的目的,但为了 CLI 的缘故,zod 仍然在清单中。

There were a bunch of examples where one npm package became one crate doing the same job. js-tiktoken became tiktoken-rs with the same o200k_base encoding. ignore became the crate of the same name with the same gitignore semantics. minimatch became globset, fast-myers-diff became similar, dompurify became ammonia, and github/keytar became keyring.有很多例子是一个 npm 包变成了执行相同工作的 crate。js-tiktoken 变成了具有相同 o200k_base 编码的 tiktoken-rs。ignore 变成了具有相同 gitignore 语义的同名 crate。minimatch 变成了 globset,fast-myers-diff 变成了 similar,dompurify 变成了 ammonia,github/keytar 变成了 keyring。

In other cases, we weren’t able to replace packages with crates one-to-one. Instead one package became several crates, or several collapsed into fewer. This is where the majority of the dependency work went. Eight opentelemetry/* packages became four crates plus a hand-written tracker state machine and file exporter. Three web-content packages, mozilla/readability, linkedom, and turndown, became two crates, readability and htmd. sharp, image-size, file-type became image and imagesize. And so on. There were also five cases where we replaced an npm package entirely with a completely custom implementation.在其他情况下,我们无法将包与 crate 一一对应地替换。相反,一个包变成了几个 crate,或者几个合并成了更少的 crate。这就是大部分依赖项工作所在。8 个 opentelemetry/* 包变成了 4 个 crate 加上一个手写的跟踪器状态机和文件导出器。3 个 Web 内容包 mozilla/readability、linkedom 和 turndown 变成了 2 个 crate:readability 和 htmd。sharp、image-size、file-type 变成了 image 和 imagesize。等等。还有 5 个例子是我们完全用自定义实现替换了 npm 包。

How much unsafe?有多少 unsafe?

Another question people ask about agent-written Rust is how much of it quietly opted out of safety guarantees. Rust’s safety is a property you can turn off with a keyword, so an agent that hits a borrow it can’t satisfy has an obvious escape hatch. Within the entire runtime crate, we now have 158 unsafe blocks, across only 36 files (alongside them are 26 unsafe fn declarations, 26 unsafe extern blocks, and nine unsafe impl trait implementations). Importantly, every single one of these is about interop with external components.人们对智能体编写的 Rust 代码提出的另一个问题是,其中有多少悄悄地放弃了安全保证。Rust 的安全性是一个可以通过关键字关闭的属性,因此无法满足借用检查的智能体有一个明显的逃生舱。在整个运行时 crate 中,我们现在有 158 个 unsafe 块,分布在 36 个文件中(旁边还有 26 个 unsafe fn 声明、26 个 unsafe extern 块和 9 个 unsafe impl trait 实现)。重要的是,每一个都与外部组件的互操作有关。

Why the unsafe block existsBlocksShare
C ABI boundary5132.3%
Windows API4931.0%
POSIX / libc4629.1%
SQLite C API74.4%
Dynamic library loading42.5%
Process environment10.6%
Donut chart of 158 unsafe blocks, all at external boundaries: C ABI, Windows API, POSIX and libc, SQLite, dynamic library loading, and process environment mutation.

The C ABI blocks are the front door SDK hosts come through, so they receive raw pointers and lengths from a caller the Rust compiler has no control over. The Windows and POSIX blocks are system calls: registry reads, credential handshakes, process trees, sysconf. SQLite is a C library. Dynamic library loading is dlopen, which cannot be safe by construction if for no other reason than because the symbol you resolve might not be the function you expected. The process environment unsafe block exists because Rust 2024 treats mutation of process-global environment state as unsafe in a multi-threaded process. Every one of those unsafe blocks marks a place where the guarantees Rust makes genuinely end: the thing on the other side is a C function, a syscall, a pointer from a foreign runtime, or process-global host state.C ABI 块是 SDK 主机进入的前门,因此它们接收来自 Rust 编译器无法控制的调用者的原始指针和长度。Windows 和 POSIX 块是系统调用:注册表读取、凭据握手、进程树、sysconf。SQLite 是一个 C 库。动态库加载是 dlopen,如果仅仅因为你解析的符号可能不是你预期的函数,它在构造上就不可能是安全的。进程环境 unsafe 块的存在是因为 Rust 2024 将多线程进程中进程全局环境状态的变动视为 unsafe。每一个 unsafe 块都标记了 Rust 所作保证真正结束的地方:另一端是 C 函数、系统调用、来自外部运行时的指针或进程全局主机状态。

The useful property is that unsafe makes all of these places in the Rust code we own auditable. The equivalent code in the TypeScript runtime crossed exactly the same boundaries, through Node’s C++ internals and native npm packages, and nothing in our source marked where the checked world stopped. Of course, that’s not a complete inventory of every safety boundary in the delivered system: dependencies, build tools, C libraries, safe wrappers, and incorrectly specified FFI contracts can still contain or expose unsafety.有用的属性是 unsafe 使我们拥有的 Rust 代码中的所有这些地方都可以审计。TypeScript 运行时中的等效代码通过 Node 的 C++ 内部和原生 npm 包跨越了完全相同的边界,而我们的源代码中没有任何标记标明检查世界在哪里停止。当然,这并不是交付系统中每个安全边界的完整清单:依赖项、构建工具、C 库、安全包装器和指定错误的 FFI 合约仍然可能包含或暴露不安全性。

The details of where unsafe is used are interesting, but I’m more interested in where it isn’t used. It’s not used in the model clients, in the MCP layer, in the agent layer, or in the prompt layer. And not one of the known regressions from the porting effort involved an unsafe block.unsafe 在哪里使用的细节很有趣,但我更感兴趣的是它在哪里没有被使用。它没有在模型客户端、MCP 层、智能体层或提示词层中使用。并且迁移工作中已知的回归中没有一个涉及 unsafe 块。

The regressions回归

Porting code is easy. Making it correct is hard. And with a codebase as large and as complicated as the Copilot agent runtime, regressions are to be expected.迁移代码很容易。使其正确很难。对于像 Copilot 智能体运行时这样庞大且复杂的代码库,回归是意料之中的。

By September 14, 2026, we’d traced dozens of known port regressions, all fixed. Most were correctness bugs, with a smaller set of performance regressions. Against the ~832,000 lines of production Rust written from scratch.到 2026 年 9 月 14 日,我们已经追踪到数十个已知的迁移回归,全部已修复。大多数是正确性错误,一小部分是性能回归。相对于从零开始编写的约 832,000 行生产级 Rust 代码。

Of course, not all of those regressions shipped. Some were caught just by developing in the repo. Others appeared in a pre-release but were fixed before making it to a stable release. And some reached a stable release, often after being unnoticeable enough to make it through one or more rounds of pre-release usage undetected.当然,并非所有这些回归都已发布。有些只是在存储库中开发时被捕获。有些出现在预发布版本中,但在进入稳定版本之前被修复。有些到达了稳定版本,通常是因为它们不够明显,以至于在一轮或多轮预发布使用中未被检测到。

Known correctness regressions grouped into five recurring failure modes: incomplete migration, state and lifetime, behavioral contract mismatches, host boundaries, and incorrect test oracles.

This isn’t zero, of course, and I’m 100% sure there are more than the ones we know about. These are the ones we noticed or that were reported, but a migration this size absolutely shipped a few more that have been quiet enough that nobody has hit them yet. As with bugs in general, I expect we’ll continue to discover a trickle of corner-case regressions as the further reaches of the stack are exercised aggressively in the wild.这当然不是零,我 100% 确定还有比我们已知的更多的回归。这些是我们注意到或被报告的,但如此规模的迁移绝对发布了一些更安静的回归,至今无人触及。正如一般的错误一样,我预计随着栈的更远部分在野外被积极使用,我们将继续发现少量边缘情况的回归。

The absolute number also isn’t all that important. What’s more important is the “why” all of these happened, so that we can learn from the issues and avoid repeating them in the future. Nearly all of the correctness regressions land in three large families: the new code implemented a different behavioral contract; state, ownership, or lifetime behavior changed; or some part of the migration was omitted, only partially applied, or lost in rebasing. A smaller set came from requirements at the host or interop boundary, and even from tests that confidently validated the wrong behavior. Those families become more concrete in a handful of recurring patterns:绝对数量也不是那么重要。更重要的是所有这些发生的“原因”,以便我们可以从问题中学习并在未来避免重复。几乎所有的正确性回归都落在三个大家族中:新代码实现了不同的行为合约;状态、所有权或生命周期行为发生了变化;或者迁移的某一部分被遗漏、仅部分应用或在变基中丢失。一小部分来自主机或互操作边界的要求,甚至来自自信地验证了错误行为的测试。这些家族在少数重复出现的模式中变得更加具体:

Ambiguous semantics. Several regressions came from behavior the source language left implicit. TypeScript has one number type; Rust requires choosing among several, including whether a value can be fractional. And the agents guess wrong. Fields that were conceptually integers became f64s, so Rust serialized values such as 42.0 instead of 42: strongly typed SDKs like Go and C# couldn’t unmarshal a repo ID into an int64 and rejected a hook timestamp and task duration. In the other direction, an agent declared timeToFirstTokenMs as i64, but the streaming path emitted values such as 5446.712845, making written sessions unreadable and unresumable. A subtler case wasn’t about types: event.error || "Unknown error" became .unwrap_or("Unknown error"). JavaScript’s || replaces an empty string; Rust’s unwrap_or preserves it, so an empty subagent error remained empty. Oops.模糊的语义。几个回归来自源语言留为隐式的行为。TypeScript 有一种 number 类型;Rust 要求在几种类型中进行选择,包括值是否可以是分数的。而智能体猜错了。概念上是整数的字段变成了 f64,因此 Rust 序列化了诸如 42.0 而不是 42 的值:像 Go 和 C# 这样强类型的 SDK 无法将存储库 ID 解组为 int64,并拒绝了钩子时间戳和任务持续时间。在另一个方向上,智能体将 timeToFirstTokenMs 声明为 i64,但流式路径发射了诸如 5446.712845 的值,使得写入的会话不可读且不可恢复。一个更微妙的情况与类型无关:event.error || "Unknown error" 变成了 .unwrap_or("Unknown error")。JavaScript 的 || 替换了空字符串;Rust 的 unwrap_or 保留了它,因此空的子智能体错误仍然为空。哎呀。

Ambient behaviors. Another recurring source was behavior JavaScript or Node supplied invisibly. Quota code used toLocaleDateString, which inherits the host time zone; Rust needed that zone passed explicitly. But although Intl.DateTimeFormat().resolvedOptions().timeZone is typed as string, it can return undefined, which napi couldn’t convert to a Rust String, breaking model-list loading. Elsewhere, moving an environment-variable read from before an await to after it meant a host change during the wait could alter the result. A native-addon loader called process.report.getReport() only to identify the platform, but on Windows that honored _NT_SYMBOL_PATH and could spend minutes downloading PDBs before rendering the CLI. Other ambient inputs included the working directory, repository identity, PATH, and session authentication. Moving ownership into Rust required deciding when to capture each one, how to carry it, and when to refresh it.环境行为。另一个反复出现的来源是 JavaScript 或 Node 不可见地提供的行为。配额代码使用了 toLocaleDateString,它继承了主机时区;Rust 需要显式传递该区域。但是,尽管 Intl.DateTimeFormat().resolvedOptions().timeZone 被类型化为 string,但它可能返回 undefined,napi 无法将其转换为 Rust String,从而破坏了模型列表加载。在其他地方,将环境变量读取从 await 之前移动到之后,意味着等待期间的主机更改可能会改变结果。一个原生插件加载器调用 process.report.getReport() 只是为了识别平台,但在 Windows 上,这遵循了 _NT_SYMBOL_PATH,并且在渲染 CLI 之前可能花费数分钟下载 PDB。其他环境输入包括工作目录、存储库身份、PATH 和会话身份验证。将所有权移动到 Rust 需要决定何时捕获每一个、如何携带它以及何时刷新它。

Only porting half of a pair. Several of the regressions came from paired operations falling out of sync. A turn-cap check updated the native registry’s abort state but didn’t cancel the in-process model loop, allowing one more request to escape. Elsewhere, task completion was persisted and emitted but not projected into active session state, so Autopilot continued after completing the task.只迁移了一对中的一半。几个回归来自配对操作不同步。turn-cap 检查更新了原生注册表的中止状态,但没有取消进程内模型循环,允许再逃逸一个请求。在其他地方,任务完成被持久化并发射,但没有投影到活跃会话状态中,因此 Autopilot 在完成任务后继续运行。

Blocking the main thread. The CLI still drives the Rust runtime from Node’s single-threaded event loop, so synchronous work across the napi boundary freezes the UI. /chronicle reindex parsed hundreds of session files this way, blocking rendering and input for nearly a minute. Making the export async and moving the work to a blocking thread-pool thread fixed it. An audit found several more potentially blocking entry points, leading to a standing rule: napi exports that do real work must be asynchronous and, when necessary, use spawn_blocking.阻塞主线程。CLI 仍然从 Node 的单线程事件循环驱动 Rust 运行时,因此跨 napi 边界的同步工作会冻结 UI。/chronicle reindex 以这种方式解析了数百个会话文件,阻塞渲染和输入近一分钟。将导出设为异步并将工作移动到阻塞线程池线程修复了它。审计发现了更多潜在的阻塞入口点,导致了一条长期规则:执行实际工作的 napi 导出必须是异步的,并且在必要时使用 spawn_blocking。

Flashing windows. On Windows, spawning a child process without CREATE_NO_WINDOW briefly opens a console window. The Node.js runtime had monkey patched process spawning to add the flag, hiding the requirement from the porting agent; the Rust replacement omitted it. An audit fixed two additional spawn sites, though those were preexisting omissions rather than port regressions, and we added the rule to the copilot instructions.闪烁的窗口。在 Windows 上,在没有 CREATE_NO_WINDOW 的情况下生成子进程会短暂打开控制台窗口。Node.js 运行时猴子补丁了进程生成以添加该标志,对迁移智能体隐藏了该要求;Rust 替换项省略了它。审计修复了另外两个生成站点,尽管那些是预先存在的遗漏而不是迁移回归,我们已将该规则添加到 copilot 指令中。

Lifecycle management. The largest cluster involved lifecycle, disposal, ownership, ordering, or races. Moving state into Rust often left TypeScript holding an opaque handle to an instance stored in a native table. Unlike an object reference, that handle can outlive the instance. One hook disposed mid-request orphaned a tool_use block, wedging the conversation because the model API required a matching result. A shell canceled between “announced” and “started” leaked an orphan that kept the session active. A sandbox toggle updated one generation counter but not its native twin, leaving the shell stuck “reconfiguring.”生命周期管理。最大的集群涉及生命周期、处置、所有权、排序或竞争。将状态移动到 Rust 通常会让 TypeScript 持有一个存储在原生表中的实例的不透明句柄。与对象引用不同,该句柄的寿命可能超过实例。一个在请求中途处置的钩子孤立了一个 tool_use 块,卡住了对话,因为模型 API 需要匹配的结果。在“宣布”和“开始”之间取消的 shell 泄漏了一个孤儿,使会话保持活跃。沙箱切换更新了一个生成计数器,但没有更新其原生孪生,导致 shell 卡在“重新配置”中。

Overlooked features. Another group involved features the port simply missed. One port omitted SDK callbacks and deleted their end-to-end test, prompting a new rule that agents must not change E2E tests without explicit consent. A session abort kept its native half but lost the in-process cancellation needed to interrupt a turn waiting in a tool. The SDK’s ability to replace built-in tool search depended on enablement, the model-facing description and schema, and routing execution to the SDK callback. The port dropped all three (yay for consistency?), silently replacing one consumer’s natural-language search with regular-expression search.被忽视的功能。另一组涉及迁移根本遗漏的功能。一个迁移省略了 SDK 回调并删除了它们的端到端测试,促使产生了一条新规则:智能体不得在未经明确同意的情况下更改 E2E 测试。会话中止保留了其原生部分,但丢失了中断等待工具的轮次所需的进程内取消。SDK 替换内置工具搜索的能力取决于启用、面向模型的描述和模式,以及将执行路由到 SDK 回调。迁移丢弃了所有三个(为一致性欢呼?),静默地将一个消费者的自然语言搜索替换为正则表达式搜索。

Different libraries with different opinions. Several more regressions came from replacing JavaScript libraries or APIs with stricter Rust equivalents. At the time, the Rust MCP SDK (rmcp) responded to malformed JSON-RPC input while the TypeScript SDK didn’t. Against a server that answered errors with more malformed output, that politeness became an infinite loop that hung startup. Related libraries in different ecosystems rarely behave identically.具有不同观点的不同库。更多回归来自用更严格的 Rust 等效项替换 JavaScript 库或 API。当时,Rust MCP SDK (rmcp) 对格式错误的 JSON-RPC 输入做出响应,而 TypeScript SDK 则没有。对于以更多格式错误输出回答错误的服务器,这种礼貌变成了一个挂起启动的无限循环。不同生态系统中的相关库很少表现得完全相同。

Ships passing in the night. A handful of regressions came from branch drift or rebasing. In a repo with hundreds of pull requests per week, a session open for days accumulates constant changes and conflicts. The port required thousands of rebases; even a very high success rate leaves failures.擦肩而过。少数回归来自分支漂移或变基。在一个每周有数百个拉取请求的存储库中,打开数天的会话会积累不断的更改和冲突。迁移需要数千次变基;即使是非常高的成功率也会留下失败。

And the slowpokes. The last group was functionally correct but slower. Some regressions lost existing efficiencies such as memoization, fully asynchronous waiting, or bounded log streaming. Others added migration-specific overhead at the Rust-TypeScript boundary: redundant serialization, locking, polling, unbounded native concurrency, and native-to-host crossings. These weren’t later optimization opportunities; each was a degradation introduced by the port. One read-only scan deep-copied a 260 MB event log instead of borrowing it. Under sustained event traffic, another implementation completed only a tiny fraction of the requested channel flushes while retaining an asynchronous handle per event until V8 exhausted its heap.还有慢动作者。最后一组在功能上是正确的,但速度较慢。一些回归丢失了现有的效率,例如记忆化、完全异步等待或有界日志流。其他在 Rust-TypeScript 边界添加了迁移特定的开销:冗余序列化、锁定、轮询、无界原生并发和原生到主机交叉。这些不是以后的优化机会;每一个都是迁移引入的退化。一个只读扫描深度复制了 260 MB 的事件日志而不是借用它。在持续的事件流量下,另一个实现仅完成了请求的通道刷新的一小部分,同时为每个事件保留了一个异步句柄,直到 V8 耗尽其堆。

Dozens of regressions sounds like a lot. But in a port that generated more than 800,000 lines of code, frankly I’m surprised and pleased we didn’t encounter an order of magnitude more. We can also get a sense for whether these were broadly “felt” by looking at trends in issues in the public github/copilot-cli and github/copilot-sdk repos. Here we’ve classified issues opened from January through August as quality related when they carried a bug label or when their title used common failure terms such as “bug,” “regression,” “crash,” “hang,” “timeout,” “broken,” or “incorrect.” The levels were essentially unchanged from before the porting work as compared to during and after it:数十个回归听起来很多。但在生成超过 800,000 行代码的迁移中,坦率地说,我感到惊讶和高兴,我们没有遇到多出一个数量级的回归。我们还可以通过查看公共 github/copilot-cli 和 github/copilot-sdk 存储库中的问题趋势来了解这些是否被广泛“感受到”。在这里,我们将 1 月到 8 月打开的问题分类为与质量相关的,如果它们带有错误标签,或者它们的标题使用了常见的失败术语,如“错误”、“回归”、“崩溃”、“挂起”、“超时”、“损坏”或“不正确”。与迁移期间和之后相比,这些水平与迁移前基本没有变化:

RepositoryJan–Apr, before the rewriteMay–Aug, during / after the rewrite
github/copilot-cli22.9% (454 / 1,982)23.7% (354 / 1,496)
github/copilot-sdk36.2% (190 / 525)32.3% (135 / 418)

That’s not an availability metric or an exact count of escaped defects… as with the regressions themselves, there can be a ton of variability in what an issue represents, its scope, and so on. But it does provide a useful check: despite the extraordinary volume of product change, the product-facing issue channels didn’t show a meaningful quality concern spike during the migration.这不是可用性指标,也不是逃逸缺陷的精确计数……与回归本身一样,问题所代表的内容、范围等可能存在巨大的差异。但它确实提供了一个有用的检查:尽管产品变更量巨大,但面向产品的反馈渠道在迁移期间没有显示出有意义的质量担忧激增。

If it compiles, it compiles如果它能编译通过,它就能编译通过

Earlier I referred to a popular meme that Rust is great for AI-generated code because of its strict compiler. There’s another related and popular Rust meme, that if the code compiles, it’s correct. Nope. Our known regression list provides a good response to that: every regression in the corpus was merged to main, which means it successfully compiled. The compiler accepted the buggy versions because, as far as the compiler was concerned, every one of them was valid Rust.早些时候,我提到了一个流行的梗,即由于 Rust 严格的编译器,它非常适合生成 AI 代码。还有另一个与之相关的流行 Rust 梗:如果代码能编译通过,那它就是正确的。并非如此。我们已知的回归列表对这一说法给出了很好的回应:语料库中的每一次回归都已合并到 main 分支,这意味着它们都成功通过了编译。编译器接受了这些有问题的版本,因为在编译器看来,每一个版本都是合法的 Rust 代码。

The compiler can prove that an f64 is used consistently. It cannot know that a repository ID must be serialized as an integer or that a timestamp serialized with a trailing .0 will be rejected by every strongly-typed SDK at the other end of the wire. The compiler can prevent unsynchronized data races in the code it can see, but it can’t prevent a perfectly synchronized state machine from encoding the wrong states. A queue can be protected by locks and still let two senders each conclude the other will drain it. Events can move safely between threads and still arrive in the wrong order. A synchronous napi function can be memory-safe and still block Node’s main thread for a minute. Compilation is also almost by definition unable to detect something that isn’t there. The compiler can’t object when a rebase quietly removes a guard and its test, or when a process spawner forgot the Windows flag that suppresses console window pop-ups. It also had no opinion about cloning a 250 MB event log on every read. A compiler checks whether the program you wrote is internally coherent. It cannot check whether you wrote the whole program, preserved the old contract, called things in the right order, met the host’s unwritten requirements, or did the work at an acceptable cost.编译器可以证明 f64 类型的使用是一致的。但它无法知道一个存储库 ID 必须序列化为整数,也无法知道一个以 .0 结尾的序列化时间戳会被另一端的每一个强类型 SDK 拒绝。编译器可以防止它所能看到的代码中出现非同步的数据竞争,但无法防止一个完全同步的状态机编码出错误的状态。一个队列可以受到锁的保护,却仍然可能让两个发送者各自认为对方会清空它。事件可以在线程之间安全地移动,但到达时顺序依然可能出错。一个同步的 napi 函数可以是内存安全的,但仍然可能阻塞 Node 的主线程长达一分钟。从定义上讲,编译器几乎无法检测出“不存在”的东西。当一次变基(rebase)悄无声息地移除了一个防护措施及其测试,或者当一个进程启动器忘记了抑制控制台窗口弹出的 Windows 标志时,编译器无法提出异议。它对于在每次读取时克隆一个 250 MB 的事件日志也没有任何意见。编译器检查的是你编写的程序在内部是否连贯。它无法检查你是否编写了整个程序、是否保留了旧的契约、是否按正确的顺序调用了事物、是否满足了宿主未明文规定的要求,或者是否以可接受的成本完成了工作。

That’s in no way an argument against Rust’s compiler. As with any statically-typed language, the compiler eliminated a huge class of mechanical mistakes and gave the agents an exceptionally useful inner loop. But “if it compiles, it’s correct” is useful only as a joke.这绝不是反对 Rust 编译器的论据。与任何静态类型语言一样,编译器消除了一大类机械错误,并为智能体提供了一个非常有用的内部循环。但“如果能编译通过,它就是正确的”这句话,只有作为笑话时才有用。

Perf, perf, and more perf性能,性能,还是性能

So what did we buy with all of this? The port was deliberately behavior-preserving. It did not set out to redesign algorithms or fix bugs; in fact, I repeatedly pushed the agents away from opportunistic optimization because changing language and behavior at the same time makes it much harder to know which one broke you. A key goal of the rewrite, however, was indeed performance and scalability (in addition to reliability and other factors). When folks asked me why did I set out to rewrite the runtime in Rust, my answer was often along the lines of “I didn’t set out to move to Rust, I set out to move away from Node.js and V8”. That move has indeed significantly moved our needle on performance.那么我们通过这一切获得了什么呢?这次移植是有意保持行为一致的。它并不打算重新设计算法或修复 Bug;事实上,我多次阻止智能体进行机会性优化,因为同时更改语言和行为会让人很难判断到底是哪一个导致了故障。然而,重写的一个关键目标确实是性能和可扩展性(除了可靠性等其他因素外)。当人们问我为什么要用 Rust 重写运行时,我的回答通常是:“我并不是为了转向 Rust,我是为了摆脱 Node.js 和 V8”。这次迁移确实显著提升了我们的性能。

I benchmarked several scenarios for the runtime via the C# SDK before and after the port (the TypeScript, Python, Go, C#, Java, and Rust SDKs all reach the same engine through the same transport architecture). The baseline against a pre-port build of the SDK and CLI, with the TypeScript runtime hosted by Node and reached over stdio. The August 21 result uses the Rust runtime, both as an out-of-process server and loaded in-process through FFI. This is an end-to-end comparison of the delivered systems rather than an attempt to isolate the effect of the language change; other changes landed during the same period, so the numbers need to be taken with some grains of salt.我在移植前后通过 C# SDK 对运行时的多个场景进行了基准测试(TypeScript、Python、Go、C#、Java 和 Rust SDK 都通过相同的传输架构连接到同一个引擎)。基准测试对比的是移植前的 SDK 和 CLI 构建版本,其中 TypeScript 运行时由 Node 托管并通过 stdio 访问。8 月 21 日的结果使用的是 Rust 运行时,既作为进程外服务器,也通过 FFI 加载到进程内。这是对交付系统的端到端比较,而不是试图孤立地衡量语言变更的影响;在此期间还有其他变更落地,因此这些数据需要谨慎解读。

Each timed turn went to a deterministic chat completion server running on localhost, producing a fixed, small response. In other words, these numbers deliberately remove model inference and network latency. They measure the part we changed: client startup, process launch, session creation, event handling, persistence, teardown, etc.每次计时轮次都针对运行在 localhost 上的确定性聊天补全服务器,产生固定的、较小的响应。换句话说,这些数据有意排除了模型推理和网络延迟。它们测量的是我们更改的部分:客户端启动、进程启动、会话创建、事件处理、持久化、销毁等。

ScenarioMay 12Aug 21 out-of-processAug 21 in-process
Client, session, one turn5.25 s1.33 s (4.0x)292 ms (18.0x)
Resume 32-turn session5.64 s1.52 s (3.7x)264 ms (21.4x)
Ten concurrent client lifecycles12.34 s4.18 s (3.0x)742 ms (16.6x)
1,000 one-turn session lifecycles132.52 s22.53 s (5.9x)20.93 s (6.3x)
From May 12 to August 21, creating a client and session, completing one turn, and tearing them down improved from 5.25 seconds to 55.3 milliseconds in-process, throughput rose from 7.55 to 120.0 sessions per second, and ten-client memory fell from 1,383 MB to 126 MB.

The “client, session, one-turn” number is the easiest one to feel. It’s creating a client, creating a session, doing a single turn, and tearing everything down. A large part of the out-of-process overhead came from starting Node, initializing V8, and loading, parsing, and generating bytecode for the JavaScript application produced from the TypeScript code before the first turn could begin. The Rust runtime removes those Node/V8 and JavaScript-loading costs.“客户端、会话、单轮”这一数据最容易被感知。它包括创建客户端、创建会话、执行单轮对话并销毁所有内容。进程外开销的很大一部分来自启动 Node、初始化 V8,以及在第一轮开始前加载、解析并为从 TypeScript 代码生成的 JavaScript 应用程序生成字节码。Rust 运行时消除了这些 Node/V8 和 JavaScript 加载成本。

The “1,000 one-turn session lifecycles” pressure test represents a step function change in what we can build. It uses a single shared client and measures running 100 concurrent pipelines, each creating a session, taking a complete model turn, disposing the session, and doing that ten times in sequence. The pre-port TypeScript CLI completed 7.55 of those lifecycles per second. Rust out-of-process completed 57.45. Rust in-process completed 120.0.“1,000 个单轮会话生命周期”压力测试代表了我们构建能力上的阶跃式变化。它使用单个共享客户端,测量运行 100 个并发管道,每个管道创建一个会话、完成一轮完整的模型对话、销毁会话,并连续重复十次。移植前的 TypeScript CLI 每秒完成 7.55 个此类生命周期。Rust 进程外模式完成了 57.45 个。Rust 进程内模式完成了 120.0 个。

That is a workload-specific result; the Rust runtime is not universally “15.9x faster.” But it is exactly the workload server hosts care about: many independent sessions sharing one runtime. And wall-clock time isn’t hiding the work on another core. In a separate resource-sampling pass over the same 100-by-10 workload, the pre-port process tree consumed 312 seconds of aggregate CPU. The Rust configurations consumed about 110 seconds. That is CPU capacity the host can spend on more sessions.这是一个针对特定工作负载的结果;Rust 运行时并非在所有情况下都“快 15.9 倍”。但这正是服务器宿主所关心的工作负载:多个独立会话共享一个运行时。而且挂钟时间并没有掩盖在另一个核心上的工作。在对同样的 100x10 工作负载进行的资源采样测试中,移植前的进程树消耗了 312 秒的总 CPU 时间。而 Rust 配置仅消耗了约 110 秒。这是宿主可以用于更多会话的 CPU 容量。

Memory tells the same story, with the usual warning that memory metrics are annoyingly easy to misuse. Looking at resident private memory added during the ten-client batch, the pre-port process tree peaked 1,383 MB above baseline. Rust out-of-process peaked at only 247 MB. And Rust in-process at 126 MB, an order of magnitude less. While these numbers of course will differ from use to use and machine to machine, they get to the core goal: a service can host far more clients and sessions on the same machine before memory, process count, or CPU become the limiting resources.内存方面的情况也类似,但需要提醒的是,内存指标非常容易被误用。观察十个客户端批处理期间增加的常驻私有内存,移植前的进程树峰值比基准高出 1,383 MB。Rust 进程外模式峰值仅为 247 MB。而 Rust 进程内模式为 126 MB,减少了一个数量级。虽然这些数字当然会因用途和机器而异,但它们触及了核心目标:服务可以在同一台机器上托管更多的客户端和会话,而不会过早受到内存、进程数或 CPU 的限制。

The best part is this is the baseline port. Much of the implementation still consists of TypeScript-shaped algorithms faithfully rendered in Rust. We haven’t yet done the broad redesign work that the new ownership model, concurrency model, and in-process architecture enable. Getting an in-process client through creation, a complete one-turn session, and teardown in about 55 milliseconds, pushing a shared client to 120 one-turn session lifecycles per second, and cutting the measured ten-client memory delta by 91% before that optimization work begins is a very good place to start.最棒的部分是,这只是基础移植。大部分实现仍然由忠实地用 Rust 呈现的 TypeScript 风格算法组成。我们尚未进行由新的所有权模型、并发模型和进程内架构所带来的广泛重构工作。在优化工作开始前,就能实现进程内客户端在约 55 毫秒内完成创建、完整单轮会话和销毁,将共享客户端推至每秒 120 个单轮会话生命周期,并将测得的十个客户端内存增量降低 91%,这是一个非常好的起点。

What the port cost移植的成本

So… how much did all this cost, monetarily? Drum roll, please….那么……这一切在金钱上花费了多少?请敲鼓……

My token spend for all of the porting work was ~136.3 billion total tokens, including ~130.6 billion cached input read tokens, ~4.2 billion cached input write tokens, ~900 million fresh input tokens, and ~600 million output tokens. The monetary bill for all those tokens came to ~$120,000.我在所有移植工作中的 Token 花费总计约为 1363 亿,其中包括约 1306 亿缓存输入读取 Token、约 42 亿缓存输入写入 Token、约 9 亿新鲜输入 Token 以及约 6 亿输出 Token。所有这些 Token 的账单总额约为 120,000 美元。

Of course, these tokens weren’t spending themselves. The cost for using a significant amount of a developer’s time guiding these agents should also be factored in. However, I wasn’t devoting all of my time just to this project. Agentic development is a lot of “hurry up and wait;” submit a prompt, let the coding agent do its thing, check in on it from time to time to possibly steer it, but in the meantime until it completes, do other things. That means developers are no longer working on just one coding task at a time; they’re overlapping that waiting to enable working on many things concurrently. During the porting window, these Rust porting PRs represented ~20% of my PRs across all repos to which I was contributing. If we wave our hands and estimate that the share of pull requests approximates the share of my time, that works out to roughly three weeks dedicated to the port.当然,这些 Token 不会自己消耗。指导这些智能体所需的开发人员时间成本也应计入。然而,我并没有将所有时间都投入到这个项目中。智能体开发包含大量的“忙碌等待”;提交提示词,让编码智能体完成它的工作,不时查看一下以进行可能的引导,但在它完成之前,可以做其他事情。这意味着开发人员不再一次只处理一项编码任务;他们将等待时间重叠,从而能够同时处理多项任务。在移植窗口期间,这些 Rust 移植 PR 约占我所有贡献仓库 PR 的 20%。如果我们粗略估计 Pull Request 的份额近似于我投入的时间份额,那么大约相当于三周的时间专门用于移植。

In other words, the rough bill for the porting effort was about $120,000 in attributed token spend plus three weeks of a developer’s time.换句话说,移植工作的粗略账单约为 120,000 美元的 Token 支出,外加三周的开发人员时间。

That said, the end-to-end Rust migration was not 100% a single developer; it’s been a team effort. @stevesandersonms provided the design and implementation for napi-oop, the temporary out-of-process interoperability layer, as well as five of the six SDK FFI implementations, with @edburns providing the sixth. @roji implemented the packaging for the SDK to ship and utilize the Rust binaries correctly. @caarlos0 has been helping to split the Rust code into many small subcrates in order to help with build times that started to become problematic, while @criemen helped with improved asset caching to speed up CI and local builds. @devm33, @examon, @MRayermannMSFT, @dereklegenzoff, and others have helped with countless pull request reviews and approvals. And everyone contributing to the copilot-agent-runtime repo has been supportive and accommodating as the world has shifted beneath their feet.话虽如此,端到端的 Rust 迁移并非完全由一名开发人员完成;这是一项团队努力。@stevesandersonms 为 napi-oop(临时的进程外互操作层)提供了设计和实现,并完成了六个 SDK FFI 实现中的五个,@edburns 提供了第六个。@roji 实现了 SDK 的打包,以便正确发布和使用 Rust 二进制文件。@caarlos0 一直在帮助将 Rust 代码拆分为许多小的子包(subcrates),以解决开始变得棘手的构建时间问题,而 @criemen 则帮助改进了资源缓存,以加速 CI 和本地构建。@devm33、@examon、@MRayermannMSFT、@dereklegenzoff 等人帮助进行了无数次的 Pull Request 审查和批准。而且,随着世界在他们脚下发生变化,每一位为 copilot-agent-runtime 仓库做出贡献的人都给予了支持和配合。

Lessons learned经验教训

The experience reinforced a few lessons that apply beyond this rewrite. Here are some things we’d carry into a next time:这次经历强化了一些适用于本次重写之外的经验教训。以下是我们下次会带入的内容:

  • The goal needs to be clearly and fully stated. Our early instructions were too vague. “Port XYZ component to Rust” got read as only the hot paths, or only the logic, and the agents repeatedly treated I/O and orchestration as being out of scope. Once we were explicit that the end state was a native binary built from a 100% Rust codebase, with no execution environment left for TypeScript even if we wanted one, they got much better at driving autonomously to that goal.目标需要清晰且完整地说明。我们早期的指令太模糊了。“将 XYZ 组件移植到 Rust”被解读为仅移植热点路径或仅移植逻辑,智能体多次将 I/O 和编排视为范围之外。一旦我们明确最终状态是一个由 100% Rust 代码库构建的原生二进制文件,且不为 TypeScript 留下任何执行环境(即使我们想要),它们在自主驱动实现该目标方面就表现得好多了。
  • End-to-end tests are absolutely, unequivocally critical. With one exception, all of the regressions that involved missing features, and many of the others, were due to lack of sufficient end-to-end tests. For any port of this nature, you must have tests that can be used to validate the correctness of the port, and those tests can’t themselves be re-written during the porting, or else you lose your oracle. Our initial plan for this porting effort called this out, and called out that we needed to significantly improve our E2E testing posture before starting on the port. We did that, but we didn’t do it enough. We’re sure that if we’d added even more E2E tests before starting the port, really focusing on ensuring most meaningful behaviors were covered by them, we would have had fewer regressions than we did along the way.端到端测试绝对至关重要。除了一次例外,所有涉及功能缺失的回归,以及许多其他回归,都是由于缺乏足够的端到端测试造成的。对于任何此类性质的移植,你必须拥有可用于验证移植正确性的测试,且这些测试本身不能在移植过程中被重写,否则你就会失去“预言机”。我们最初的移植计划提到了这一点,并指出在开始移植之前需要显著改进我们的 E2E 测试状况。我们确实做了,但做得不够。我们确信,如果我们在开始移植前添加更多的 E2E 测试,真正专注于确保大多数有意义的行为都被覆盖,我们在过程中遇到的回归会更少。
  • Protect the oracle from the agent. The agent changing an implementation cannot also be allowed to silently redefine correctness by weakening a test, updating a snapshot, raising a compatibility baseline, or applying an escape-hatch label, at least not without oversight. Keep the behavioral contract independent where possible, put sensitive guardrails behind separate ownership or approval, and layer checks with different failure modes so that one mistake isn’t enough to ship a big regression.保护“预言机”不受智能体影响。不能允许智能体在更改实现的同时,通过削弱测试、更新快照、提高兼容性基准或应用逃生舱标签来悄悄重新定义正确性,至少不能在没有监督的情况下这样做。尽可能保持行为契约的独立性,将敏感的护栏置于单独的所有权或审批之下,并分层设置具有不同失败模式的检查,这样单一的错误就不至于导致重大的回归发布。
  • Translate first, redesign second. Preserving behavior and existing algorithms kept the number of variables moving at once manageable. Once the old implementation and the transitional scaffolding are gone, ownership, concurrency, and performance can be redesigned against a stable baseline. I veered away from this a few times, out of restlessness or difficulty saying no to a peer or a conviction that this case was different, and in hindsight I regret every one of them. Each cost more regressions, more time, or more tokens than staying the course would have.先翻译,后重构。保持行为和现有算法不变,使得同时变动的变量数量在可控范围内。一旦旧的实现和过渡性的脚手架消失,就可以在稳定的基准上重新设计所有权、并发性和性能。我曾几次偏离这一点,出于焦躁、难以拒绝同行,或者坚信这个案例与众不同,事后看来,我后悔每一次偏离。每一次偏离所带来的回归、时间或 Token 成本,都比坚持原有路线要高。
  • Turn repeated failures into future successes. AI agents will wander off track: when they do, learn from it. When a failure mode shows up twice, it belongs in standing instructions, a reusable skill, an eval, a protected baseline, or the harness itself.将重复的失败转化为未来的成功。AI 智能体会偏离轨道:当它们偏离时,要从中吸取教训。当某种失败模式出现两次时,它就应该被纳入常设指令、可重用技能、评估、受保护的基准或测试框架本身中。
  • The developer inner loop matters more with agents in the loop, not less. For us as developers, we live and breathe in our inner loop, how quickly we can make a change, build, test, iterate. We get frustrated when the tooling part of that takes too long. And it’d be forgivable to think that doesn’t apply when agents are doing the lower-level work. It does. More so. AI agents do the thinking and writing code parts of these tasks incredibly quickly, but they still need to build and they still need to test. And the proportion of the time they build and test actually increases as they spend less time thinking and writing and more time in fast validation inner loops. Spend some time up front optimizing the inner loop, and optimizing it for multiple things happening at the same time (e.g. as if you’re working on multiple tasks in multiple worktrees at the same time). You will thank yourself later for the investment.当智能体参与其中时,开发人员的内部循环更为重要。作为开发人员,我们生活在内部循环中,即我们进行更改、构建、测试和迭代的速度。当其中的工具部分耗时过长时,我们会感到沮丧。如果认为当智能体在做底层工作时这就不适用了,那是可以原谅的。但事实并非如此。情况恰恰相反。AI 智能体在进行这些任务的思考和编写代码部分时速度极快,但它们仍然需要构建和测试。随着它们花在思考和编写上的时间减少,花在快速验证内部循环上的时间增加,它们构建和测试的时间比例实际上在上升。花点时间预先优化内部循环,并针对同时发生的多项任务进行优化(例如,就像你同时在多个工作树中处理多项任务一样)。你会为这一投资而在未来感谢自己。

What’s next?接下来是什么?

It worked. The execution runtime that was entirely TypeScript in May is now entirely Rust in August, and it shipped to real users continuously the whole way rather than landing as one terrifying cutover at the end.它成功了。5 月份完全是 TypeScript 的执行运行时,在 8 月份已经完全变成了 Rust,并且在整个过程中持续交付给真实用户,而不是在最后进行一次可怕的切换。

I didn’t simply ask an AI agent to “port this whole codebase from TypeScript to Rust.” Even if that’s where we’re heading as an industry, we’re definitely not there yet. Instead, agents made an entire category of project feasible. A rewrite producing hundreds of thousands of lines of production Rust in a live system, done in place, in main, by one engineer supported by a team, is not a proposal that would have been accepted before agents. It would have needed a whole team and a year or two, it would have competed against every feature that team could have shipped instead, and it would have lost (and, honestly, should have lost). Agents moved the price to where the project became tenable.我并没有简单地要求 AI 智能体“将整个代码库从 TypeScript 移植到 Rust”。即使这是我们行业的发展方向,我们也绝对还没有达到那个阶段。相反,智能体使一整类项目变得可行。在一个活跃系统中进行重写,产生数十万行生产级 Rust 代码,在 main 分支中原地完成,由一名工程师在团队支持下执行,这在智能体出现之前是不可能被接受的提议。它需要整个团队花费一两年时间,它会与该团队本可以交付的每一个功能竞争,并且会失败(老实说,也应该失败)。智能体将成本降低到了项目变得可行的程度。

The port itself is complete: the runtime’s production implementation is 100% Rust, and the temporary internal TypeScript/N-API seam is gone. There is still plenty of work we desire to do, though: further improving the build system and developer inner loop, cleaning up translated structures, redesigning around Rust’s ownership and concurrency models, and pursuing further performance wins. The port was a translation, deliberately so (prompted as such), and we kept behavior as close to 100% identical as possible, rather than opportunistically fixing additional bugs, rearchitecting components, or further improving performance and scalability (beyond what came implicitly from the rewrite). Much of the code at the micro level is idiomatic Rust, but at the macro level there’s a good deal of initially TypeScript algorithms wearing Rust’s syntax. Revisiting those decisions now that the constraints underneath them have changed is where the interesting wins will be.移植本身已经完成:运行时的生产实现 100% 是 Rust,临时的内部 TypeScript/N-API 缝合层已经消失。不过,我们仍有许多想做的工作:进一步改进构建系统和开发人员内部循环,清理翻译后的结构,围绕 Rust 的所有权和并发模型进行重新设计,并追求进一步的性能提升。这次移植是一次翻译,是有意为之的(按照提示进行),我们尽可能保持行为 100% 一致,而不是机会主义地修复额外的 Bug、重构组件或进一步提升性能和可扩展性(超出重写本身带来的隐性提升)。微观层面的大部分代码是地道的 Rust,但在宏观层面,仍有大量最初的 TypeScript 算法披着 Rust 的语法。现在底层的约束已经改变,重新审视这些决策将是获得更有趣成果的地方。

I’m most excited about what the port makes possible. The SDK can be loaded directly into a host process in any of six languages, with no Node.js or V8 in the dependency chain, and no second process to supervise, which is the single most common piece of friction we’ve heard from partners adopting the SDK. A runtime instance that costs a fraction of what it used to means a host can run far more concurrent sessions before it runs out of machine. And the runtime can now go places Node.js was never going to follow it, across the spectrum of cloud to desktop to device to embedded systems. None of that is the finish line. It’s the base we now get to build the future of GitHub Copilot on top of, and after three months of watching agents rewrite the very engine that runs them, I’m looking forward to seeing just how far it goes.我最兴奋的是这次移植所带来的可能性。SDK 可以直接加载到六种语言中任一语言的宿主进程中,依赖链中没有 Node.js 或 V8,也不需要第二个进程来监管,这是我们从采用 SDK 的合作伙伴那里听到的最常见的摩擦点。一个成本仅为过去一小部分的运行时实例意味着宿主在耗尽机器资源前可以运行更多的并发会话。而且,运行时现在可以进入 Node.js 从未涉足的地方,横跨从云端到桌面、设备再到嵌入式系统的整个范围。这并不是终点。这是我们现在得以构建 GitHub Copilot 未来的基础,在观察了三个月智能体重写运行它们的核心引擎后,我很期待看到它能走多远。

Happy coding!祝编码愉快!

Tags:标签:

Written by 作者

Related posts