You can’t be model agnostic if you’re hand-tuning prompts如果你在手动调优提示,就无法对模型保持中立

Thanks to natural language interfaces, AI applications can be prototyped quickly. You write what you want in English, hand it to a frontier model, and a working prototype appears in an afternoon. This is extraordinarily powerful and for one-off tasks, optimal. But as a way to build reliable systems, the natural language prompt is a trap.得益于自然语言接口,AI 应用可以快速原型化。你用英语写下需求,交给前沿模型,一个可工作的原型就在下午出现。这种力量非同寻常,对于一次性任务来说是最佳方案。但作为构建可靠系统的方式,自然语言提示是一种陷阱。

The plain-English prompt that makes prototypes effortless turns out to be a poor way to specify how a system should behave, and the bill arrives slowly, disguised as ordinary progress, until the application can barely move. The problem is not any single prompt. It is that natural language was never meant to be a specification language for engineering, and treating it as one quietly caps what you can build.看似轻松的英文提示让原型变得毫不费力,却是指定系统行为的糟糕方式,账单会慢慢到来,伪装成普通的进展,直到应用几乎动弹不得。问题不在于某个单一提示,而是自然语言本就不适合作为工程规范语言,而把它当作规范语言会悄悄限制你能构建的东西。

The Prompt Debt Trap提示债务陷阱

The first symptom of prompt debt is slowing iteration. As users flag errors and spot edge cases, additional guidance is added to the instructions, nudging the model into line. If unwanted behaviors persist, instructions are repeated, with increasing severity. Pretty soon, the prompt isn’t straightforward and quick fixes regress previous instructions. Errors can no longer be handled with one-line “hot fixes” and your development cycle slows to a crawl.提示债务的第一个症状是迭代变慢。当用户标记错误并发现边缘案例时,额外的指导会被加入指令中,推动模型走向正确。如果不良行为仍然存在,指令会被重复,且语气越来越严厉。很快,提示不再简洁,快速修复会导致之前的指令回退。错误再也无法用一行“热修复”解决,开发周期慢到爬行。

Fable's system prompt repeats copyright guidance up to six times, with increasing severity

Fable's system prompt repeats copyright guidance up to six times, under sections named search_instructions, search_usage_guidelines, mandatory_copyright_requirements, hard_limits, self_check_before_responding, and critical_reminders.Fable 的系统提示在名为 search_instructions、search_usage_guidelines、mandatory_copyright_requirements、hard_limits、self_check_before_responding 和 critical_reminders 的章节中重复了版权指导多达六次。

Next, prompt debt incapacitates your team. Your brittle prompt full of edge cases and all-caps threats is barely legible to you, and it’s downright impenetrable to your colleagues. Many teams mitigate this issue by breaking prompts into complicated templates assembled at run-time, each isolated to specific concerns. But these prompt segments evolve, too, growing into a thicket of conditions.接下来,提示债务使你的团队失能。你那充满边缘案例和全大写威胁的脆弱提示对你来说几乎难以辨认,对同事更是彻底不可理解。许多团队通过将提示拆分为在运行时组装的复杂模板来缓解此问题,每个模板针对特定关注点。但这些提示片段也会演变,最终变成一团错综复杂的条件。

Finally, prompt debt ties you to a single model. Your hot fixes work on GPT-4o, but fail in entirely new ways when you point your inference call at GPT-5.4-mini. So you stay with 4o, hope the increasingly frequent deprecation emails from your inference provider are empty threats, and forgo the possibility of potentially cheaper, faster, better models. A recent report from Datadog suggests this is a common situation: the most-used model in traffic they observed is GPT-4o1.最终,提示债务把你锁定在单一模型上。你的热修复在 GPT-4o 上有效,但当你把推理调用指向 GPT-5.4-mini 时会以全新的方式失效。因此你只能继续使用 4o,寄希望于推理提供商日益频繁的弃用邮件只是空洞威胁,并放弃可能更便宜、更快、更好的模型。Datadog 最近的报告显示,这种情况很常见:他们观察到的流量中使用最广的模型是 GPT-4o1。

Any one of these issues is a nuisance, but together they are the difference between a glorified prototype and a product that can grow with you, your customers, and your business. Your shiny new AI features are frozen, can only be improved through a full rebuild, and are locked to an aging model.任意一个问题都是烦恼,但它们合在一起就决定了是光鲜的原型还是能够随你、你的客户和你的业务一起成长的产品。你那闪亮的新 AI 功能被冻结,只能通过完整重建来改进,并且被锁定在老旧模型上。

Why Prompt Debt Happens提示债务为何会出现

Natural language interfaces are wonderful. They’re the right mechanism for one-off tasks and broad conversational threads. We get into trouble when we rely on natural language to define durable system behavior.自然语言接口很棒。它们是一次性任务和宽泛对话的正确机制。当我们依赖自然语言来定义持久系统行为时,就会陷入困境。

The imprecision of natural language paired with probabilistic language models means different words expressing the same intent, can yield different outputs. In a recent study, a clinical question asked in a patient’s voice and then re-asked in a physician’s, with identical facts, flipped Opus from declining all ten times to answering all ten.自然语言的模糊性加上概率语言模型的特性意味着,用不同的词表达相同意图,可能产生不同的输出。在最近的一项研究中,同一个临床问题用患者的口吻提问,再用医生的口吻重新提问,事实完全相同,却把 Opus 从十次全部拒答翻转为十次全部回答。

And it’s not only word choice that matters. Seemingly unrelated statements, in the same prompt, can affect results. In a Harvard study, researchers found that merely stating which NFL team the user rooted for changed how often the model refused to answer questions regarding sensitive topics. Spurious statements influence the inference pass in ways we can’t predict. Which is why prompts become more brittle as you add fixes. An additional instruction to quell a stubborn error could affect how the model interprets a separate instruction that worked yesterday.而且不仅仅是用词会影响结果。同一提示中看似无关的陈述也会影响结果。在哈佛的一项研究中,研究人员发现,仅仅说明用户支持的 NFL 球队,就会改变模型拒答敏感话题的频率。无关的陈述以我们无法预测的方式影响推理过程。这就是为什么随着你添加修复,提示会变得更脆弱。为了解决顽固错误而添加的额外指令,可能会影响模型对昨天还能正常工作的另一条指令的解释。

Repeating instructions propels us towards prompt debt, but it’s necessary when the behavior we want is at odds with a model’s training. This is fighting the weights, and once you recognize it you see it in system prompts everywhere. For example, ChatGPT’s image prompts used to instruct the LLM eight times to not reply when a generated image was returned, because it had been trained to always keep the conversation going.重复指令会把我们推向提示债务,但当我们想要的行为与模型的训练相冲突时,这种重复是必要的。这是在与权重作斗争,一旦你意识到这一点,就会在各处的系统提示中看到它。例如,ChatGPT 的图像提示曾经八次指示大语言模型在返回生成图像时不要回复,因为它被训练成始终保持对话进行。

Every coding agent system prompt we analyzed featured repeated instructions, stern warnings, and all-caps demands. Claude Code tells Opus seven times to return multiple tool calls in a single response. And even the most advanced models force prompt authors to fight the weights: Fable’s leaked system prompt restates one specific copyright rule six times.我们分析的每个编码代理系统提示都包含重复指令、严厉警告和全大写的要求。Claude Code 七次告诉 Opus 在单个响应中返回多个工具调用。即使是最先进的模型也迫使提示作者与权重作斗争:Fable 泄露的系统提示将同一条版权规则重复了六次。

None of these examples occurred in isolation. Multiple repeated rules are woven throughout the system prompts we examine. Stubborn errors grow our prompts quickly, with each increasing the brittleness, the risk of regression with every edit.这些例子并非孤立出现。我们审查的系统提示中交织着多条重复规则。顽固错误让我们的提示快速膨胀,每一次添加都增加脆弱性,并在每次编辑时带来回归风险。

And worse: these fixes are tailored to a single model’s behavior. A recent Berkeley-led study found enterprises stay on older models because newer ones break their existing agents. This is because models are not cleanly versioned software. They have different weights that produce different behaviors, in unpredictable and undocumented ways. A prompt that works beautifully with GPT-4o may fail with GPT-5.5. Anthropic’s own release notes for Fable warn that skills developed for prior models can “degrade output quality”.更糟的是:这些修复针对单一模型的行为量身定制。最近一项由伯克利主导的研究发现,企业仍停留在旧模型上,因为新模型会破坏它们现有的代理。这是因为模型并不像软件那样有清晰的版本,它们的权重不同,导致行为以不可预测且未记录的方式变化。一个在 GPT-4o 上表现完美的提示,在 GPT-5.5 上可能会失效。Anthropic 对 Fable 的发布说明也警告说,针对旧模型开发的技能可能会“降低输出质量”。

Prompt debt locks an application to a single model. Our inability to easily swap models isn’t the result of frontier labs coming up with a clever moat. No, it’s the result of evolving a lossy, natural language specification against a probabilistic model.提示债务把应用锁定在单一模型上。我们难以轻松切换模型并不是前沿实验室设下的巧妙壁垒,而是因为在概率模型上演化出一种有损的自然语言规范。

Preventing Prompt Debt防止提示债务

Thankfully, we don’t have to theorize about how to mitigate prompt debt; one field has already shown the way. Programmers using coding agents sit at the leading edge of what models can do, outliers on the jagged frontier of model abilities. Over the last couple years they’ve been evolving best practices that let the model write more of the code, while delivering maintainable, modular software.值得庆幸的是,我们不必去理论化如何缓解提示债务;已有一个领域展示了路径。使用编码代理的程序员站在模型能力的前沿,是模型能力锯齿形前沿的异常者。过去几年,他们一直在演进最佳实践,让模型编写更多代码,同时交付可维护、模块化的软件。

The first principle is to specify your system’s behavior with measurements, not prose. When the model’s output is probabilistic and language is imprecise, we build hard edges to constrain them: evaluations, metrics, and typed specifications. These are legible, shared artifacts colleagues can read and contribute to, enabling the collaboration that brittle prompts prevented.第一原则是用度量而非散文来指定系统行为。当模型输出是概率性的、语言又不精确时,我们构建硬边界来约束它们:评估、指标和类型化规范。这些是可读、可共享的产物,同事可以阅读并贡献,从而实现脆弱提示阻碍的协作。

The best engineers now spend more of their bandwidth on tests than ever, as they are no longer a safety net but the thing that lets the model cook.现在最优秀的工程师把更多带宽花在测试上,因为测试不再是安全网,而是让模型真正发挥作用的关键。

The second principle is to stop writing the prompt by hand. Once we have metrics that can score candidates, the prompt is no longer something to craft but something for which to search. And the surface area of potential words, phrases, and structures that natural language allows is too vast to spend human hours on. This is terrain LLMs were built to explore, and there are already systems (like DSPy and GEPA) that manage this work for you, holding prompts accountable to your designs.第二原则是停止手工编写提示。一旦我们拥有可以对候选进行评分的指标,提示就不再是需要精心打造的东西,而是需要搜索的对象。自然语言允许的词汇、短语和结构的潜在表面空间太大,无法让人类花费数小时去遍历。这正是大语言模型被构建来探索的领域,已有系统(如 DSPy 和 GEPA)可以为你管理这项工作,使提示对你的设计负责。

Once prompts are generated and your program’s behavior is defined by measurements, you are no longer bound to a particular model. Evaluating a new model takes hours, not weeks. When a faster, cheaper model arrives you can try it. When a deprecation email arrives, you can secure options in a day. Whether a model is pulled for regulatory reasons (as we saw with Anthropic’s Fable) or deprecated due to age (as Groq announced last week with Llama-3.1-8b), the fix is a chore, not a fire drill.一旦提示被生成,程序行为由度量定义,你就不再受限于特定模型。评估新模型只需数小时,而非数周。当更快、更便宜的模型出现时,你可以立即尝试。收到弃用邮件时,你可以在一天内确定备选方案。无论是因监管原因被下架(如我们看到的 Anthropic 的 Fable),还是因老化被弃用(如 Groq 上周宣布的 Llama-3.1-8b),修复都只是日常事务,而不是紧急演练。

Every mature engineering discipline eventually stops doing by hand the very thing it once prided itself on doing by hand. Assembly gave way to compilers, hand-tuned queries gave way to planners, and manual memory management gave way (mostly) to machines that do it better. Prompt-writing is no different.每个成熟的工程学科最终都会停止手工完成曾经以手工自豪的工作。装配让位于编译器,手工调优查询让位于规划器,手动内存管理让位(大多数情况下)于做得更好的机器。提示编写也不例外。

Coaxing the model with exactly the right words is a real skill, and for one-off tasks it’s often optimal. But to build reliable, improvable, and portable systems we should not be hand-tuning prompts.用恰当的词语引导模型是一项真实的技能,对于一次性任务它往往是最佳方案。但要构建可靠、可改进且可移植的系统,我们不应手动调优提示。


  1. This stat from Datadog is from March of this year, so GPT-4o concentration has likely dropped a bit. However, I’ve heard from multiple large inference providers that usage of GPT-4o and models of similar vintage can be higher than 50% of all calls! 这条来自 Datadog 的统计数据是今年三月的,所以 GPT-4o 的占比可能已经略有下降。不过,我从多家大型推理提供商那里听说,GPT-4o 及类似年代的模型的使用率可能超过所有调用的 50%! ↩