Automating eval design and hillclimbing with Claude

Evaluations provide a signal on how your app or skill is performing on specific tasks. But designing evaluations, and improving performance on them without fooling yourself, is hard. We've added guidance for both to the claude-api skill.评估可以为你提供关于应用程序或技能在特定任务上表现的信号。但设计评估，并在不自欺欺人的前提下提升表现，是一件困难的事。我们已在 claude-api 技能中为这两者添加了指导。

With the skill, you can run /claude-api build-eval to build an evaluation inside your codebase, and run /claude-api hillclimb to improve your application against it, one change at a time, with a held-out set of examples to catch overfitting.借助该技能，你可以运行 /claude-api build-eval 在代码库中构建评估，并运行 /claude-api hillclimb 来针对评估逐步优化你的应用程序。通过使用一组预留的示例，你可以有效防止过拟合。

In this article, we highlight the principles of good eval design and hillclimbing first, then show how Claude Code with the claude-api skill applies those principles. We’ll close by showing a few examples of these commands.在本文中，我们将首先强调优秀评估设计和爬坡（hillclimbing）的原则，然后展示 Claude Code 如何通过 claude-api 技能应用这些原则。最后，我们将通过几个示例来演示这些命令的使用。

EVAL DESIGN

Well designed evaluations have a few common elements (Figure 1):设计良好的评估通常具备以下几个共同要素（图 1）：

1. Eval tasks mirror production. Sample tasks that you care about in “production,” or the setting in which the capability or application you are testing will be used. Sometimes tasks are picked because they are easy to generate or they are easy to grade. But it’s important to ensure that the task distribution represents what you actually care about.评估任务应反映生产环境。选取你在“生产环境”中关心的任务样本，即你所测试的能力或应用程序实际运行的环境。有时选择某些任务是因为它们易于生成或评分，但确保任务分布能够代表你真正关心的内容至关重要。
2. Performance improves with stronger models and more thinking. More capable models and higher effort levels typically should perform better on an evaluation. If they don’t, ambiguous tasks or a miscalibrated grader often are hobbling performance.性能随模型能力的增强和思考投入的增加而提升。更强大的模型和更高的努力程度通常应在评估中表现更好。如果表现没有提升，往往是因为任务模糊或评分器校准不当阻碍了性能发挥。
3. There is “passable” headroom at the frontier. The most capable model at the highest effort should be well below 100% on the evaluation, otherwise you can’t reliably judge how changes impact performance. Importantly, the gap should not be explained by impossible or ambiguous tasks: a common tell is that a task fails every evaluation run, regardless of the number of replicates. A good task is one where two domain experts would reach the same verdict and everything the grader checks is stated in the task.前沿模型应留有“可提升”的空间。最强模型在最高努力程度下，其评估得分应明显低于 100%，否则你无法可靠地判断改动对性能的影响。重要的是，得分差距不应由不可能完成或模糊的任务导致：一个常见的迹象是，无论重复多少次，某个任务在每次评估运行时都会失败。好的任务应该是两位领域专家能达成一致结论，且评分器检查的所有内容都在任务中明确说明。
4. Low run-to-run variance. High variance is often due to poorly designed, ambiguous tasks or a grader that produces different verdicts on identical output. Variance can also hide in the configuration. For example, effort may not be applied consistently. Also, the environment can affect the results of the evaluation: leftover state from an earlier trial (a file, a git history) can hand the agent the answer.低运行方差。高方差通常源于设计糟糕、模糊的任务，或者评分器对相同输出产生不同判定。方差也可能隐藏在配置中。例如，努力程度可能没有被一致地应用。此外，环境也会影响评估结果：前一次试验留下的状态（如文件、Git 历史记录）可能会直接将答案泄露给智能体。

Adversarial sampling对抗性采样

Model capability is jagged. If you pick cases because today's model fails them, you are sampling the valleys of one model's capability surface (Figure 2). The evaluation can end up measuring that model's failure fingerprint rather than what is intrinsically hard or valuable for your application to do.模型能力是参差不齐的。如果你因为当前模型在某些案例上失败而选择它们，你实际上是在采样该模型能力表面的低谷（图 2）。评估最终衡量的可能是该模型的失败指纹，而非对你的应用程序而言本质上困难或有价值的内容。

Pick hard cases because a human judged them hard: a useful test is to be able to say why a task is hard before you include it. Include cases that are specific failures in your application derived from production traffic, bug reports, or tickets. However, don’t blindly trust user traffic: users sometimes try what they expect to work, so a task distribution drawn strictly from user traffic may skew easy.选择人类判断为困难的案例：一个有效的测试方法是，在包含某个任务之前，你能说出它为什么困难。包含那些源自生产流量、错误报告或工单的特定失败案例。然而，不要盲目信任用户流量：用户有时会尝试他们预期能成功的事，因此完全从用户流量中提取的任务分布可能会偏向简单。

/CLAUDE-API BUILD-EVAL/CLAUDE-API BUILD-EVAL

The build-eval command in the claude-api skill turns these principles into a guided workflow. When you run /claude-api build-eval in Claude Code, Claude interviews you, builds the eval inside your codebase, and pauses for approval at specific points.claude-api 技能中的 build-eval 命令将这些原则转化为引导式工作流。当你运行 /claude-api build-eval 时，Claude 会通过询问来了解你的需求，在你的代码库中构建评估，并在特定节点暂停以等待你的批准。

Designing examples设计示例

Claude helps you sample inputs to build evaluations in this order:Claude 会按以下顺序协助你采样输入以构建评估：

1. Production transcripts, after asking about retention and sensitive data.生产环境转录，前提是已询问过关于留存和敏感数据的问题。
2. Bug reports and support tickets.错误报告和支持工单。
3. Five to ten cases you write by hand.你手动编写的五到十个案例。
4. Cases synthesized from your codebase.从你的代码库中合成的案例。

The skill prioritizes production traffic, but it can also generate synthetic data anchored in a few real examples that you provide. The skill instructs Claude to generate a simple page that shows you every input and waits until you confirm them. As an illustration, below we show an example set of inputs for an e-mail router application that the skill may ask the user to review (Figure 3).该技能优先处理生产流量，但也能基于你提供的少量真实示例生成合成数据。该技能会引导 Claude 生成一个简单的页面向你展示所有输入，并等待确认。作为说明，下面展示了一个电子邮件路由应用程序的输入示例集，技能可能会要求用户对此进行审查（图 3）。

Validating the grader验证评分器

After the inputs, Claude proposes the cheapest grader that fits your application’s output:输入确定后，Claude 会提出最适合你应用程序输出的低成本评分方案：

• Programmatic verification: If the output possibilities are constrained, it uses a code based check (exact match, a label from a fixed set, JSON that matches a schema, tests that pass).程序化验证：如果输出可能性受限，它会使用基于代码的检查（精确匹配、固定集合中的标签、符合模式的 JSON、通过的测试等）。
• LLM-as-judge: It will default to this type of check if the output space is open-ended, with many valid answers but clear quality criteria. In this case, a second model reads the input, the output and a rubric written as checkable claims (not a 1-to-5 scale), and returns a score with its reasoning. If you have a baseline to compare against, the judge instead reads both inputs in random order, without being told which is the baseline, and picks the better one. You pick the judge model, and it should not be the model you are testing.LLM 作为评判者：如果输出空间是开放式的，且存在多种有效答案但有明确的质量标准，它将默认使用这种检查方式。在这种情况下，第二个模型会读取输入、输出以及以可检查声明（而非 1 到 5 的评分标准）编写的准则，并返回带有推理过程的评分。如果你有基准可供对比，评判者则会读取随机顺序的两个输入（不告知哪个是基准），并选出更好的一个。你负责选择评判模型，且该模型不应与你正在测试的模型相同。

Claude grades a handful of cases and asks whether you would have scored any of them differently (Figure 4). In general, it is important to read a sample of scored transcripts before believing your evaluator; scoring failures are among the most common ways an evaluation is misconfigured.Claude 会对少量案例进行评分，并询问你是否会对其中任何案例给出不同的分数（图 4）。通常，在信任你的评估器之前，阅读一部分已评分的转录内容非常重要；评分失败是评估配置错误最常见的原因之一。

When you’ve validated the grader, the skill tells you the size of the evaluation set (cases × repeats × model, and roughly how long it will take), runs the baseline, and prints the score with a confidence interval. What you get back: the cases, the grader, the runner, one JSON line and one full transcript per case, and a plain page that lists each case's score with a link to its transcript. If you want more than that page shows (e.g., a chart), just ask and Claude will build it as an extra page next to it. By default, these extra pages are static files that open locally and load nothing from the network.当你验证完评分器后，该技能会告知你评估集的大小（案例数 × 重复次数 × 模型，以及大致耗时），运行基准测试，并打印带有置信区间的得分。你将获得：案例、评分器、运行器、每个案例一行 JSON 和一份完整转录，以及一个列出每个案例得分并附带转录链接的纯文本页面。如果你需要页面显示之外的内容（例如图表），只需提出要求，Claude 就会在旁边额外构建一个页面。默认情况下，这些额外页面是本地打开且不从网络加载任何内容的静态文件。

Diagnostic checks诊断检查

During the baseline runs mentioned above, Claude checks a number of things:在上述基准运行期间，Claude 会检查以下几项：

• Grader: Claude runs the grader twice on the same output, and reports whether the verdict changed.评分器：Claude 会对同一输出运行两次评分器，并报告判定结果是否发生变化。
• Plumbing: Claude checks for timeouts, API errors, and cut-off answers to ensure infrastructure noise doesn't pass as model variance.基础设施：Claude 会检查超时、API 错误和截断的回答，以确保基础设施噪声不会被误认为是模型方差。
• Headroom: if the baseline already scores about 95% or higher, the skill warns the user and alerts that the hillclimb should aim to explore cost or latency rather than quality.提升空间：如果基准测试得分已达到约 95% 或更高，该技能会向用户发出警告，提示爬坡应侧重于探索成本或延迟，而非质量。

HILLCLIMBING

Now that you have a reliable means of grading your application’s performance on a task, you can try to improve it. Hillclimbing is an effective way to tune parameters like effort or prompts, which trade-off cost and performance. Some general tips for choosing where to apply it:现在你已经有了可靠的方法来评估应用程序在某项任务上的表现，可以尝试对其进行改进。爬坡是调整努力程度或提示词等参数的有效方式，这些参数在成本和性能之间进行权衡。关于选择应用场景的一些通用建议：

• Cheap iteration - It should be inexpensive (in terms of time, cost, and effort) to modify whatever surface you are focused on for hillclimbing. Many internal efforts and customers have focused hillclimbing on text, such as prompts and skills. These are easy to change and revert. In contrast, open-ended modifications to an agent harness during hillclimbing may involve extensive code changes.低成本迭代 - 修改你关注的爬坡对象时，成本（时间、金钱和精力）应较低。许多内部项目和客户都将爬坡重点放在文本上，例如提示词和技能。这些内容易于更改和回滚。相比之下，在爬坡过程中对智能体工具链进行开放式修改可能涉及大量的代码变更。
• Attributable - Changes in the score on your evaluation should be attributable to the surface you are modifying during hillclimbing. For example, several successful applications of hillclimbing have focused on skill triggering. The evaluation metric (the trigger rate for the skill) is directly coupled to the skill description that is being modified.可归因性 - 评估得分的变化应能归因于你在爬坡过程中修改的对象。例如，一些成功的爬坡应用集中在技能触发上。评估指标（技能触发率）与正在修改的技能描述直接相关。
• Well-scoped objective - One common failure mode is an open-ended request to improve performance without careful consideration of the headroom available in the evaluation; an evaluation that’s near saturation or a poorly scoped surface (e.g., an open-ended request to update the harness) is more likely to stall. One generally strong objective across various efforts is cost: even if an evaluation is saturated, you can ask Claude to find ways to reduce cost while keeping performance at parity.目标明确 - 一个常见的失败模式是盲目要求提升性能，而没有仔细考虑评估中的可用空间；如果评估已接近饱和，或者修改对象界定不清（例如，笼统地要求更新工具链），则更容易陷入停滞。在各种项目中，一个普遍强有力的目标是成本：即使评估已饱和，你也可以要求 Claude 在保持性能持平的同时寻找降低成本的方法。

Overfitting过拟合

Even a well-designed evaluation rarely matches the exact task distribution you care about in production. As a result, "overfitting" to an evaluation is a common problem and results in a system that performs better on an evaluation than on production traffic.即使设计良好的评估也极少能完全匹配你在生产环境中关心的任务分布。因此，“过拟合”到评估是一个常见问题，会导致系统在评估中的表现优于生产流量中的表现。

There are many ways an evaluation can "leak" into your harness (the code around the model, including prompts, tools, and loop that calls Claude). For example, consider an evaluation task that benefits from OCR, but OCR is rarely beneficial in your production tasks. The evaluation harness might add an OCR tool to your application, which improves on the benchmark without any impact on production. More broadly, hillclimbing may add features to that harness that address edge cases in the particular evaluation examples you’ve chosen. These harness additions improve your evaluation score, but don’t translate to improvements in production (Figure 5).评估可能会通过多种方式“泄露”到你的工具链（模型周围的代码，包括提示词、工具和调用 Claude 的循环）中。例如，考虑一个从 OCR 中受益的评估任务，但在你的生产任务中 OCR 很少有用。评估工具链可能会向你的应用程序添加一个 OCR 工具，这在基准测试上有所改进，但对生产环境没有任何影响。更广泛地说，爬坡可能会向工具链添加针对你所选特定评估示例中边缘情况的功能。这些工具链的添加提高了评估得分，却无法转化为生产环境的改进（图 5）。

Three things can help address this:三件事有助于解决这个问题：

• Split the cases. Use a train set that the hillclimber may read and a test set that is never seen. If the train set scores improve while the test set scores stay flat, then that is a common overfitting warning sign.拆分案例。使用爬坡器可以读取的训练集，以及从未见过的测试集。如果训练集得分提高而测试集得分保持平稳，这就是一个常见的过拟合警告信号。
• Never paste failures into the prompt. If the hillclimber reads the failing transcripts, it should never paste the failure content into the prompt.切勿将失败案例粘贴到提示词中。如果爬坡器读取了失败的转录内容，它绝不应将失败内容粘贴到提示词中。
• Keep the answers structurally out of the model's reach. Models can sometimes “reward hack” by directly finding answers to evaluations.使答案在结构上处于模型无法直接获取的位置。模型有时会通过直接找到评估答案来“奖励黑客行为”。

As discussed below, the claude-api skill applies these principles for you.如下文所述，claude-api 技能会为你应用这些原则。

/CLAUDE-API HILLCLIMB/CLAUDE-API HILLCLIMB

The hillclimb command in the claude-api skill turns these principles into a guided workflow. When you run /claude-api hillclimb in Claude Code, Claude iterates to improve against a given evaluation. You choose what changes it can make including:claude-api 技能中的 hillclimb 命令将这些原则转化为引导式工作流。当你运行 /claude-api hillclimb 时，Claude 会进行迭代以针对给定的评估进行改进。你可以选择它能进行的更改，包括：

• Your system prompt你的系统提示词
• Skills or instruction files技能或指令文件
• Tool descriptions工具描述
• Model choice, effort level, and other API parameters模型选择、努力程度及其他 API 参数
• Your harness code你的工具链代码

Before it starts, Claude asks what you want to optimize (e.g., performance, or cost while performance holds) and then splits the evaluation set at random into test and train. With a cost goal, it considers a few common cost drivers, including prompt caching, auditing the prompt for compatibility with the selected model, and picking the model and effort setting.在开始之前，Claude 会询问你想要优化的目标（例如，性能，或在性能保持不变的情况下降低成本），然后随机将评估集拆分为测试集和训练集。对于成本目标，它会考虑几个常见的成本驱动因素，包括提示词缓存、审计提示词以确保与所选模型的兼容性，以及选择模型和努力程度设置。

Before the first round, Claude checks that the eval's noise (how far the score can move by chance alone) is smaller than the smallest improvement you'd act on; if it isn't, it says so and suggests more repetitions or cases.在第一轮之前，Claude 会检查评估噪声（仅凭偶然因素得分可能变动的范围）是否小于你准备采取行动的最小改进幅度；如果不满足，它会说明并建议增加重复次数或案例数量。

Each round, Claude reads the previous round’s train transcripts and proposes one change as a patch. It aims each round at a change whose effect can show above the eval's noise: it fixes the failing behavior at its root (e.g., rewrites the section that causes it or adds a missing rule) rather than rewording a line. It then runs evaluation with the patched change. At this point, Claude applies a check: if the train set improves but the test set is flat, Claude suspects overfitting and reverts the patch. If there is a regression, Claude reverts. If train and test sets improve, it keeps the patch (Figure 6).每一轮，Claude 都会读取上一轮的训练转录，并提出一个补丁作为更改。每一轮的目标都是一个其效果能超越评估噪声的更改：它从根本上修复失败行为（例如，重写导致失败的部分或添加缺失的规则），而不是仅仅修改一行措辞。然后，它会带着补丁运行评估。此时，Claude 会执行检查：如果训练集得分提高但测试集持平，Claude 会怀疑过拟合并回滚补丁。如果出现回归，Claude 也会回滚。如果训练集和测试集均有所提高，它则保留该补丁（图 6）。

When the score stalls for two or three rounds, Claude reads each remaining train failure and sorts it by cause. It does the same early if no single fix could gain more than the eval's noise, and suggests more repetitions or cases, rather than spending rounds on changes too small to measure. This step can catch ambiguous evaluation cases, harness errors, or run-to-run variance.当得分连续两到三轮停滞时，Claude 会读取剩余的每个训练失败案例，并按原因进行分类。如果没有任何单一修复能带来超过评估噪声的收益，它会提前执行此操作，并建议增加重复次数或案例数量，而不是将轮次浪费在难以衡量的微小改动上。这一步可以捕捉到模糊的评估案例、工具链错误或运行方差。

Only legitimate failures are included in more hillclimbing rounds.只有真正的失败案例才会被纳入更多的爬坡轮次中。

When hillclimbing completes, Claude leaves your code at the version that did best on the test set for your goal. It reports the test result against the baseline with confidence intervals (Figure 7). If the gain is within noise, it says so and recommends against merging.爬坡完成后，Claude 会将你的代码保留在测试集上表现最好的版本。它会报告测试结果与基准对比的置信区间（图 7）。如果增益在噪声范围内，它会说明这一点并建议不要合并。

EXAMPLES

Hillclimbing for cost reduction针对降低成本的爬坡

We ran /claude-api hillclimb on an internal customer support benchmark with the goal of reducing cost and improving performance. The benchmark included 44 tickets, with 30 used for the search and 14 held out. It started on Opus 4.8 at default (high) effort settings with 74.4% decision accuracy on the search tickets and a token cost of 4.6 cents per ticket.我们在一个内部客户支持基准测试上运行了 /claude-api hillclimb，目标是降低成本并提升性能。该基准测试包含 44 个工单，其中 30 个用于搜索，14 个预留。它最初使用 Opus 4.8，在默认（高）努力程度设置下，搜索工单的决策准确率为 74.4%，每个工单的 Token 成本为 4.6 美分。

The hillclimb first audited the prompt, removing mandatory tool-call rituals, a scratchpad step, and contradictory rules. Then it tried Opus 5.5 on low effort. This cleared the baseline accuracy bar at 87.8% and cut cost to 1.9 cents per ticket, less than half the starting cost.爬坡器首先审计了提示词，移除了强制性的工具调用仪式、草稿步骤和相互矛盾的规则。然后，它尝试了 Opus 5.5 的低努力程度设置。这使准确率达到了 87.8%，超过了基准线，并将每个工单的成本降至 1.9 美分，不到初始成本的一半。

Part of that saving comes from Opus 5.5's pricing: input and output tokens cost 20% less than on Opus 4.8, and cache reads cost 60% less. Because Opus 5.5 cleared the bar, the hillclimb then stepped down a tier to check whether a cheaper model could clear it too. Sonnet 5 on low effort scored about the same, 88.9%, at about half the cost, 1 cent per ticket (Figure 8).部分节省来自 Opus 5.5 的定价：输入和输出 Token 成本比 Opus 4.8 低 20%，缓存读取成本低 60%。由于 Opus 5.5 达标，爬坡器随后下调了一个级别，以检查更便宜的模型是否也能达标。Sonnet 5 在低努力程度下得分约为 88.9%，成本约为原来的一半，即每个工单 1 美分（图 8）。

Finally, Claude improved the prompt with routing rules and a refund-cap cross-reference, bringing Sonnet 5 to 98.9% at about the same cost. On the 14 held-out tickets that the search never saw, the final configuration scored 90.5% against the original setup's 78.6%, at about one fifth of the cost.最后，Claude 通过路由规则和退款上限交叉引用改进了提示词，使 Sonnet 5 在成本大致相同的情况下达到了 98.9% 的准确率。在从未见过的 14 个预留工单上，最终配置的得分为 90.5%，而原始设置仅为 78.6%，且成本约为原来的五分之一。

Hillclimbing for performance improvement针对性能提升的爬坡

Another example is our claude-api skill, which provides guidance on using our APIs and general tips for working with Claude (including the sub-commands discussed in this article). We want to ensure our skill can correctly implement code that uses our APIs, and we built an evaluation set derived from our documentation to test the skill.另一个例子是我们的 claude-api 技能，它提供有关使用我们 API 的指导以及使用 Claude 的通用技巧（包括本文中讨论的子命令）。我们希望确保我们的技能能够正确实现使用我们 API 的代码，因此我们根据文档构建了一个评估集来测试该技能。

On our evaluation, the skill started at 66%. We gave the hillclimber access to documentation and our SDKs, allowing Claude to identify errors and self-correct them (Figure 9). Claude found that the skill was missing coverage of eight features.在我们的评估中，该技能起步得分为 66%。我们让爬坡器访问文档和我们的 SDK，使 Claude 能够识别错误并自我纠正（图 9）。Claude 发现该技能缺少对八项功能的覆盖。

Adding sections for them in the skill improved performance to 74%. It then found errors in C# and Java type tables, boosting performance to 77%.在技能中为这些功能添加章节后，性能提升至 74%。它随后发现了 C# 和 Java 类型表中的错误，将性能提升至 77%。

After the score stalled for two rounds, Claude analyzed the remaining failures and bucketed them by root-cause. A normal round makes one edit for the most common failure. This step makes no edit; it only sorts every remaining failure by cause. This reflection step was useful in a few ways:在得分停滞两轮后，Claude 分析了剩余的失败案例并按根本原因进行归类。普通轮次会针对最常见的失败进行一次编辑。这一步不进行编辑，只按原因对所有剩余失败进行排序。这种反思步骤在几个方面很有用：

• Reflecting across a collection of failures, the hillclimber found that the skill content was present but Claude was simply writing older API shapes (e.g., from its trained priors). To address, the hillclimber added a table near the top of the skill that guided Claude from the forms it remembered to the current ones: for example, from extended thinking with a fixed token budget, which the API now rejects on recent Opus models, to adaptive thinking, and from older versions of the web search and web fetch tools to the current ones. It also moved the C# and Java warnings against fixed-budget thinking above their adaptive-thinking examples. This improved performance to 80%.通过对一系列失败进行反思，爬坡器发现技能内容本身是存在的，但 Claude 只是在编写旧版的 API 形式（例如，基于其训练先验）。为了解决这个问题，爬坡器在技能顶部附近添加了一个表格，引导 Claude 从它记忆中的形式转向当前形式：例如，从 API 在较新的 Opus 模型上不再支持的固定 Token 预算的扩展思考，转向自适应思考；从旧版本的网页搜索和网页抓取工具转向当前版本。它还将 C# 和 Java 中针对固定预算思考的警告移到了自适应思考示例之上。这使性能提升到了 80%。
• Tasks that never improved in performance despite addressing obvious content gaps are tells that the example or grader is flawed. One task asked for code that catches one error type, while its grader wanted a chain of at least three. Claude reworded the task. Another grader's instructions contradicted our docs, and testing the real API showed the docs were right. Addressing these, along with more skill edits, brought performance to ~88%.尽管解决了明显的内容缺失，但性能始终没有提升的任务，是示例或评分器存在缺陷的信号。一个任务要求编写捕获一种错误类型的代码，而其评分器却要求至少捕获三种。Claude 重写了该任务。另一个评分器的说明与我们的文档相矛盾，而测试真实 API 表明文档是正确的。解决这些问题，加上更多的技能编辑，使性能达到了约 88%。

GETTING STARTED

These sub-commands can be used directly in Claude Code via the claude-api skill:这些子命令可以通过 claude-api 技能直接在 Claude Code 中使用：

Run /claude-api build-eval if you want to generate an evaluation set for a particular problem. You can steer it by providing access to examples (e.g., traces). Claude will employ the guidance shared in this article to design the examples and grader, and ensure you approve the examples and the grader.如果你想为特定问题生成评估集，请运行 /claude-api build-eval。你可以通过提供示例（例如追踪记录）来引导它。Claude 将采用本文分享的指导原则来设计示例和评分器，并确保你批准这些示例和评分器。

Run /claude-api hillclimb if you have an evaluation and want Claude to improve on this, guided by your goal (e.g., better performance, or lower cost while performance holds). Claude will employ the guidance shared in this article to check for overfitting while climbing and check for bugs in the eval itself, such as a grader that marks a correct-looking answer wrong or a harness error, both before the first round and whenever the score stalls.如果你已经有一个评估集并希望 Claude 在此基础上进行改进，请运行 /claude-api hillclimb，并设定你的目标（例如，更好的性能，或在性能保持不变的情况下降低成本）。Claude 将采用本文分享的指导原则，在爬坡过程中检查过拟合，并在第一轮之前以及每当得分停滞时，检查评估本身是否存在错误（例如评分器将正确的答案标记为错误，或工具链错误）。

With special thanks to Misha Khalman for skill development. With thanks to Misha Khalman, Michael Segner, Matt Bell, and Matt Thanabalan for reviews, contributions, and product support.特别感谢 Misha Khalman 在技能开发方面的贡献。感谢 Misha Khalman、Michael Segner、Matt Bell 和 Matt Thanabalan 的评审、贡献和产品支持。