How to Build a Model Router in the Harness如何在智能体框架中构建模型路由

Step 1: understand the tasks第一步：理解任务

Our test bed was Open SWE, which our engineers use from Slack and a web UI to ask questions about our codebases and request code changes. Before building a router, we needed to understand the types of tasks that developers use Open SWE for. We pulled thread-level data from LangSmith traces: the kinds of requests coming in, plus the cost and turn count of each thread (as approximate measures of complexity). We did the exploration in LangSmith Custom Apps, with a small interface built directly on top of the Open SWE traces.我们的测试平台是 Open SWE，工程师们通过 Slack 和 Web 界面使用它来查询代码库并请求代码变更。在构建路由之前，我们需要了解开发者使用 Open SWE 处理的任务类型。我们从 LangSmith 追踪记录中提取了线程级别的数据：包括传入的请求类型，以及每个线程的成本和轮次计数（作为复杂度的近似度量）。我们使用直接构建在 Open SWE 追踪记录之上的小型界面，在 LangSmith 自定义应用（Custom Apps）中完成了这项探索。

We took a week of interactive threads and labeled each one by task type using an LLM classifier. LangSmith Insights can also do this kind of grouping across your traces for you. Code changes dominated: new features (22%) and bug fixes (17%) were the two largest groups, followed by test or no-op runs (16%). Categories are heuristics based on each thread's title and metadata.我们选取了一周的交互线程，并使用 LLM 分类器按任务类型对每个线程进行了标记。LangSmith Insights 也可以为你完成这种跨追踪记录的分组。代码变更占据主导地位：新功能（22%）和错误修复（17%）是最大的两个类别，其次是测试或无操作运行（16%）。这些类别是基于每个线程的标题和元数据得出的启发式分类。

We also investigated how agent trace characteristics differed based on task type. We discovered that threads associated with feature work investigations tended to be longer, with higher cost and median turn count. Threads associated with testing and release procedures tended to be relatively short and inexpensive.我们还调查了智能体追踪特征如何随任务类型而变化。我们发现，与功能调研相关的线程往往更长，成本和中位数轮次计数更高。而与测试和发布流程相关的线程则相对较短且成本较低。

In order to judge “complexity”, we used both total cost and invocation as signals: cost tracks complexity fairly directly, since bigger tasks use more tokens, while invocations are more nuanced, since a high count can mean a harder task or a model that needed follow-ups to accomplish the task.为了判断“复杂度”，我们同时使用了总成本和调用次数作为信号：成本相当直接地反映了复杂度，因为更大的任务需要消耗更多 Token；而调用次数则更为微妙，因为高调用次数可能意味着任务更难，或者模型需要多次跟进才能完成任务。

At the time of data collection, all of the Open SWE threads were being routed through a top-tier frontier model. The above data suggested that many of the tasks Open SWE handled might not require frontier intelligence, given the range of complexities.在数据收集时，所有的 Open SWE 线程都通过顶级前沿模型进行路由。上述数据表明，考虑到复杂度的范围，Open SWE 处理的许多任务可能并不需要前沿智能。

This variation in task complexity gave us a hypothesis worth testing: a router could infer a task’s type and difficulty from the initial request and send it to a cheaper or faster model without degrading outcomes.这种任务复杂度的差异给了我们一个值得验证的假设：路由可以从初始请求中推断任务的类型和难度，并将其发送给更便宜或更快的模型，而不会降低结果质量。

Step 2: understand the models第二步：理解模型

The Artificial Analysis Intelligence Index scores models on a common set of tasks and reports the cost per task, so you can plot them all on one curve of intelligence against cost. The Pareto frontier is the set of models that are the cheapest and smartest.Artificial Analysis Intelligence Index 在一组通用任务上对模型进行评分并报告每项任务的成本，因此你可以将它们绘制在同一条智能水平对成本的曲线上。帕累托前沿（Pareto frontier）是指那些在成本和智能水平上最优的模型集合。

We decided on three models along the curve, each with a different balance of cost, speed, and intelligence:我们决定沿着曲线选择三个模型，每个模型在成本、速度和智能水平上都有不同的平衡：

• Fast: GLM-5.3-Flash (xhigh)快速型：GLM-5.3-Flash（高）
• Balanced: GPT-5.6 Sol (medium)均衡型：GPT-5.6 Sol（中）
• Performance: GPT-6 Astra (low)性能型：GPT-6 Astra（低）

We selected models from different providers, and the fast tier is an open model. GLM-5.3-Flash sits on the Pareto frontier next to closed models, one more sign that open models have crossed a threshold. LangChain is model agnostic, with a generic model interface that works the same across providers, so when a better model lands, swapping it in is a one-line change to the router.我们选择了来自不同提供商的模型，且快速层级采用的是开源模型。GLM-5.3-Flash 位于帕累托前沿上，与闭源模型并列，这再次证明开源模型已经跨越了一个门槛。LangChain 是模型无关的，拥有通用的模型接口，可以在不同提供商之间无缝切换，因此当更好的模型出现时，只需修改一行代码即可替换路由中的模型。

Step 3: build the router in the harness第三步：在框架中构建路由

With the task mix mapped and three tiers picked, we then need to match the tasks to models by reading each incoming request and sending it to the cheapest tier that's able to address it successfully. In LangChain, that decision fits naturally in middleware, which can swap the model an agent calls without changing anything else about the agent (see dynamic model selection).在完成任务组合映射并选定三个层级后，我们需要通过读取每个传入的请求，将其发送给能够成功处理该请求的最便宜层级，从而实现任务与模型的匹配。在 LangChain 中，这一决策自然地融入了中间件，它可以在不改变智能体其他任何配置的情况下替换所调用的模型（参见动态模型选择）。

The router in Open SWE runs on the thread’s first human message. It has three parts:Open SWE 中的路由在线程的第一个人类消息上运行。它包含三个部分：

• A base prompt: tells the classifier its job: pick the least expensive model likely to complete the task.基础提示词：告知分类器其任务：选择最有可能完成任务且成本最低的模型。
• Criteria per tier: a short, plain-language description of the work each tier should take.各层级标准：用简短、通俗的语言描述每个层级应承担的工作。
• A classifier model: reads the request and picks a tier according to the criteria and prompt.分类器模型：读取请求并根据标准和提示词选择一个层级。

A general benchmark is only a starting point. Write each tier's criteria from two sources: your own task analysis, and what each provider says its models are best at. We combined the task breakdown from step 1 with the provider guides for GPT-5.6 Sol, GPT-6 Astra, and GLM-5.3-Flash to write the base prompt and the criteria for each tier.通用基准只是一个起点。请结合两个来源来编写每个层级的标准：你自己的任务分析，以及每个提供商宣称其模型最擅长的领域。我们结合了第一步中的任务分解与 GPT-5.6 Sol、GPT-6 Astra 和 GLM-5.3-Flash 的提供商指南，编写了基础提示词和每个层级的标准。

Those criteria are written for Open SWE's task set, so the router is deeply coupled to the tasks Open SWE handles. That’s why it belongs in the harness, which already has the agent’s task-specific context (its prompt, tools, and domain knowledge) that a generic gateway lacks.这些标准是针对 Open SWE 的任务集编写的，因此路由与 Open SWE 处理的任务紧密耦合。这就是为什么它属于框架的原因，因为框架已经具备了智能体特定任务的上下文（其提示词、工具和领域知识），而通用网关则缺乏这些。

Our first version used an LLM with structured output, prompted with the user's request. The classifier now runs on Jev, a newly released decision model, which made classification almost 50× faster. See how we did this in Building a Harness with Jev.我们的第一个版本使用了带有结构化输出的 LLM，并以用户请求作为提示。现在的分类器运行在 Jev（一种新发布的决策模型）上，这使得分类速度提高了近 50 倍。请参阅《构建带有 Jev 的框架》了解我们是如何做到的。

The router picks a model once, at the start of each thread, and that model is used for the whole thread. The natural objection is: what if a thread changes topic or complexity mid-flight? Though not addressed in this naive router design, we cover mid-flight routing in the “what’s next” section below.路由在每个线程开始时选择一次模型，该模型将用于整个线程。一个自然的异议是：如果线程在运行中途改变了主题或复杂度怎么办？虽然这种简单的路由设计没有解决这个问题，但我们将在下文的“后续计划”部分讨论中途路由。

Step 4: track task outcomes第四步：跟踪任务结果

The value in a router is lower cost, but only if quality doesn't regress. A router has to get two things right: the chosen model needs to be able to complete the task, and it should be the cheapest, fastest model that can. That means you need a way to track task outcomes. There are two ways to do this:路由的价值在于降低成本，但前提是质量不能倒退。路由必须做好两件事：所选模型必须能够完成任务，并且它应该是能完成任务的最便宜、最快的模型。这意味着你需要一种跟踪任务结果的方法。有两种方法可以实现这一点：

1. Offline evals run the router against a fixed dataset, so you can compare versions safely and repeatably. The catch is the dataset: it has to look like your real traffic and be graded on what your users care about. For a coding agent, that means PR quality and reviewability, which are hard to grade offline.离线评估（Offline evals）在固定数据集上运行路由，以便安全且可重复地比较版本。难点在于数据集：它必须看起来像你的真实流量，并根据用户关心的指标进行评分。对于编码智能体而言，这意味着 PR 的质量和可审查性，这些在离线状态下很难评分。
2. An A/B test splits live threads between the router and a single-model baseline and compares them on a success metric you can measure for every thread. The live traffic is naturally a representative dataset, but the downside here is that users are subjected to a potentially non-optimal router.A/B 测试将实时线程分配给路由和单一模型基准，并比较它们在每个线程上可测量的成功指标。实时流量自然是一个具有代表性的数据集，但缺点是用户可能会体验到非最优的路由。

We ran A/B tests with two outcome signals:我们使用两个结果信号进行了 A/B 测试：

• Merged PRs: Open SWE now records every PR it opens and whether it's merged or closed. Merged PRs per thread became our main success metric.合并的 PR：Open SWE 现在记录它打开的每个 PR 以及它是被合并还是关闭。每个线程的合并 PR 数成为我们的主要成功指标。
• User feedback: we added thumbs up and down to Open SWE, so users could rate any thread, including questions that never produce a PR. Each rating is logged as feedback on the thread's LangSmith trace.用户反馈：我们在 Open SWE 中添加了点赞和点踩功能，以便用户可以对任何线程进行评价，包括那些从未产生 PR 的问题。每个评分都会记录为该线程 LangSmith 追踪记录上的反馈。

We tracked both in a LangSmith Custom App. Merged PRs were the stronger signal. Feedback was sparse, since only a small share of threads get rated, but it's where we heard about routing mistakes, like the ones quoted in the first test below.我们在 LangSmith 自定义应用中跟踪了这两项指标。合并的 PR 是更强的信号。反馈比较稀疏，因为只有一小部分线程会被评分，但我们正是通过这里听到了关于路由错误的反馈，例如下面第一个测试中引用的那些。

The experiment实验

Our first A/B test compared the router against always using our strongest model: half of threads went through the router, and half always used GPT-6 Astra, across 973 threads in total.我们的第一次 A/B 测试将路由与始终使用我们最强模型进行了对比：总共 973 个线程中，一半通过路由，另一半始终使用 GPT-6 Astra。

Quality didn't measurably change. 29.2% of routed threads ended in a merged PR vs. 27.3% of control (p = 0.49). PR open rates were also flat (38.9% vs. 39.6%, p = 0.82).质量没有出现可测量的变化。路由线程的 PR 合并率为 29.2%，对照组为 27.3%（p = 0.49）。PR 打开率也持平（38.9% 对 39.6%，p = 0.82）。

Cost dropped a lot. The median routed thread cost $0.94 vs. $2.61 on control, 64% less. The mean dropped 42% and the p90 dropped 37%, so the savings weren't just a few cheap outliers.成本大幅下降。路由线程的中位数成本为 0.94 美元，而对照组为 2.61 美元，降低了 64%。平均值下降了 42%，p90 下降了 37%，因此节省的成本不仅仅来自少数几个廉价的离群值。

Most requests didn't need the strongest model. Of routed threads, 56% went to balanced, 34% to fast, and only 10% to performance. The cost ladder between tiers is steep: the median thread cost $0.097 on fast, $1.50 on balanced, and $2.88 on performance, a 30× spread.大多数请求并不需要最强的模型。在路由线程中，56% 分配给均衡型，34% 分配给快速型，只有 10% 分配给性能型。各层级之间的成本梯度非常陡峭：中位数线程成本在快速型上为 0.097 美元，均衡型上为 1.50 美元，性能型上为 2.88 美元，差距达 30 倍。

User feedback pointed the same way. When a simple request ran on GPT-6 Astra, engineers flagged the overspend directly with comments like: "pretty expensive for this query" and "this request should not have been routed to the performance model."用户反馈也指向了同样的方向。当一个简单的请求在 GPT-6 Astra 上运行时，工程师直接指出了超支问题，评论如：“这个查询太贵了”和“这个请求不应该被路由到性能模型”。

💡 This result isn't surprising, given the control was our most expensive model. But it's a change a lot of agents would benefit from: many over-rotate on quality and end up overspending. Especially for agents with a diverse set of tasks, routing can drive cost down.💡 考虑到对照组是我们最昂贵的模型，这一结果并不令人惊讶。但这是一个许多智能体都可以从中获益的改变：许多智能体在质量上过度投入，最终导致超支。特别是对于任务多样的智能体，路由可以显著降低成本。

We also tested the router against the opposite control: half of threads went through the router, and half always used the fast model. We ended this test within a day, before it could produce statistically meaningful results. Engineers flagged problems with the fast-only arm almost immediately, and it was disrupting their productivity due to low output quality.我们还针对相反的对照组测试了路由：一半线程通过路由，另一半始终使用快速模型。我们在一天之内就结束了这项测试，还没来得及得出具有统计学意义的结果。工程师们几乎立即就指出了“仅限快速模型”组的问题，由于输出质量低下，它干扰了他们的工作效率。

What's next后续计划

This implementation of a model router is a proof of concept that serves as a foundation for building an optimal router. Here are a few areas we could explore to improve:这种模型路由的实现是一个概念验证，为构建最优路由奠定了基础。以下是我们未来可以探索以进行改进的几个方向：

• Benchmarking the router on DeepSWE or other coding benchmarks, so we can evaluate routing decisions against a controlled baseline instead of relying on A/B testing on production traffic alone.在 DeepSWE 或其他编码基准上对路由进行基准测试，以便我们可以根据受控基准来评估路由决策，而不是仅依赖生产流量上的 A/B 测试。
• Model selection for subagents. Right now subagents pick their model independently of the router. Routing subagents too, especially on longer-running tasks, could cut costs further.子智能体的模型选择。目前子智能体独立于路由选择模型。对子智能体也进行路由，特别是在长时间运行的任务中，可以进一步削减成本。
• Re-routing mid-thread. When a new message differs enough from earlier ones, like a question that turns into a bug fix, a model change might pay off. The cost is the prompt cache: switching models throws it away, so the new model re-reads the thread at full price. For async agents that cost is often negligible, since the cache often expires anyways between human turns when the TTL is short (like 5 minutes).线程中途重新路由。当新消息与之前的消息差异足够大时（例如一个问题变成了错误修复），更换模型可能会带来收益。成本在于提示词缓存：切换模型会丢弃缓存，因此新模型需要以全价重新读取线程。对于异步智能体，这种成本通常微不足道，因为当 TTL 很短（如 5 分钟）时，缓存通常在人类轮次之间就已经过期了。
• Refining the routing criteria with more signals, such as mining for user sentiment in traces: finding threads where users are frustrated, which can mean the task needed a stronger model.利用更多信号优化路由标准，例如挖掘追踪记录中的用户情绪：寻找用户感到沮丧的线程，这可能意味着该任务需要更强的模型。

Getting started入门指南

If you're adding routing to your own agent, here's where we'd start:如果你正在为自己的智能体添加路由功能，以下是我们的建议步骤：

1. Understand the tasks. Your traces hold the real record of what your agent is asked to do, and LangSmith Insights helps you find the patterns in them, so you can see the task mix before you pick tiers.理解任务。你的追踪记录真实记录了智能体被要求做的事情，LangSmith Insights 可以帮你发现其中的模式，从而在选择层级之前看清任务组合。
2. Understand the models. Pick a few models along the cost-intelligence curve, informed by modern benchmarks. LangChain's model interface works across providers, so you can swap models at any time without having to re-engineer your application.理解模型。参考现代基准测试，沿着成本-智能曲线选择几个模型。LangChain 的模型接口适用于所有提供商，因此你可以随时更换模型，而无需重新设计你的应用程序。
3. Build the router in the harness. Treat routing as context engineering, analogous to classical feature engineering: decide what information the router should see in order to make the best decision about model fit given your agent's domain.在框架中构建路由。将路由视为上下文工程，类似于传统的特征工程：决定路由为了根据你的智能体领域做出最佳模型匹配决策，需要看到哪些信息。
4. Track task outcomes. Put measures of success in place before you route: evals, online evaluators, or user feedback on traces. If building an eval dataset is too costly or difficult, an A/B test on live traffic works well.跟踪任务结果。在路由之前建立衡量成功的标准：评估、在线评估器或追踪记录上的用户反馈。如果构建评估数据集太昂贵或困难，在实时流量上进行 A/B 测试效果很好。

The right model for a task keeps changing as new models, open-weight ones included, keep landing on the frontier. Because LangChain is model agnostic, picking up those gains means swapping a tier, not rebuilding your agent.随着新模型（包括开源权重模型）不断出现在前沿，任务的最佳模型也在不断变化。因为 LangChain 是模型无关的，利用这些进步意味着更换层级，而不是重建你的智能体。

To add routing to your own agent, you can add our newly released model routing middleware. Give it your base prompt, model tiers, and the criteria for each, and it picks a model at the start of every thread. We’d love your feedback on X or LangChain GitHub issues.要为你的智能体添加路由，可以使用我们新发布的模型路由中间件。提供你的基础提示词、模型层级和每个层级的标准，它会在每个线程开始时选择一个模型。我们期待你在 X 或 LangChain GitHub issue 上提供反馈。