AI Is Writing Our Code Faster Than We Can Verify ItAI 正在比我们验证代码更快地编写代码

This is the fourth article in a series on agentic engineering and AI-driven development. Read part one here, part two here, part three here, and look for the next article on April 30 on O’Reilly Radar.这是关于代理工程和 AI 驱动开发系列的第四篇文章。请在此处阅读第一部分，在此处阅读第二部分，在此处阅读第三部分，下一篇文章将于 4 月 30 日在 O’Reilly Radar 上发布。

Here’s the dirty secret of the AI coding revolution: most experienced developers still don’t really trust the code the AI writes for us.AI 编码革命的肮脏秘密是：大多数有经验的开发者仍然不太信任 AI 为我们编写的代码。

If I’m being honest, that’s not actually a particularly well-guarded secret. It feels like every day there’s a new breathless “I don’t have a lick of development experience but I just vibe coded this amazing application” article. And I get it—articles like that get so much engagement because everyone is watching carefully as the drama of AIs getting better and better at writing code unfolds. We’ve had decades of shows and movies, from WarGames to Hackers to Mr. Robot, portraying developers as reclusive geniuses doing mysterious but incredible stuff with computers. The idea that we’ve coded ourselves out of existence is fascinating to people.说实话，这并不是一个特别保密的秘密。每天似乎都有新的“我没有任何开发经验，却凭感觉写出了这个惊人应用”的文章。我能理解——这类文章之所以引起大量关注，是因为大家都在密切关注 AI 在写代码方面越来越强的戏剧性进展。我们已经有了数十部电视剧和电影，从《战争游戏》到《黑客》再到《黑客帝国》，把开发者描绘成隐居的天才，用电脑做神秘而惊人的事。我们自己把代码写得让自己消失的想法对人们很有吸引力。

The flip side of that pop-culture phenomenon is that when there are problems caused by agentic engineering gone wrong (like the equally popular “I trusted an AI agent and it deleted my entire production database” articles), everyone seems to find out about it. And, unfortunately, that newly emerging trope is much closer to reality. Most of us who do agentic engineering have seen our own AI-generated code go off the rails. That’s why I built and maintain the Quality Playbook, an open-source AI skill that uses quality engineering techniques that go back over fifty years to help developers working in any language verify the quality of their AI-generated code. I was as surprised as anyone to discover that it actually works.这种流行文化现象的另一面是，当代理工程出错（比如同样流行的“我信任了 AI 代理，它删掉了我的整个生产数据库”文章）时，大家都会得知。而且不幸的是，这种新出现的情节更接近现实。大多数从事代理工程的人都见过自己的 AI 生成代码失控。这就是我创建并维护 Quality Playbook 的原因——一个开源 AI 技能，使用超过五十年的质量工程技术，帮助任何语言的开发者验证 AI 生成代码的质量。我和其他人一样惊讶于它真的有效。

I’ve talked often about how we need a “trust but verify” mindset when using AI to write code. In the past, I’ve mostly focused on the “trust” aspect, finding ways to help developers feel more comfortable adopting AI coding tools and using them for production work. But I’m increasingly convinced that our biggest problem with AI-driven development is that we don’t have a reliable way to check the quality of code from agentic engineering at scale. AI is writing our code faster than we can verify it, and that is one of AI’s biggest problems right now.我经常谈到在使用 AI 编写代码时需要“信任但要验证”的心态。过去，我主要关注“信任”方面，寻找方法让开发者更愿意采用 AI 编码工具并在生产环境中使用。但我越来越确信，AI 驱动开发最大的问题是我们没有可靠的方法在大规模上检查代理工程代码的质量。AI 正在比我们验证更快地编写代码，这正是 AI 目前最大的难题之一。

A false choice错误的选择

After I got my first real taste of using AI for development in a professional setting, it felt like I was being asked to make a critical choice: either I had to outsource all of my thinking to the AI and just trust it to build whatever code I needed, or I had to review every single file it generated line by line.在职业环境中首次真正体验到使用 AI 开发后，我感觉自己被迫做出关键选择：要么把所有思考外包给 AI 并完全信任它构建所需代码，要么逐行审查它生成的每个文件。

A lot of really good, really experienced senior engineers I’ve talked to feel the same way. A small number of experienced developers fully embrace vibe coding and basically fire off the AI to do what it needs to, depending on a combination of unit tests and solid, decoupled architecture (and a little luck, maybe) to make sure things go well. But more frequently, the senior, experienced engineers I’ve talked to, folks who’ve been developing for a really long time, go the other way. When I ask them if they’re using AI every day, they’ll almost always say something like, “Yeah, I use AI for unit tests and code reviews.” That’s almost always a tell that they don’t trust the AI to build the really important code that’s at the core of the application. They’re using AI for things that won’t cause production bugs if they go wrong.我交谈过的许多优秀且经验丰富的高级工程师也有同感。少数经验丰富的开发者完全拥抱“感觉编码”，基本上让 AI 按需工作，依赖单元测试和稳固、解耦的架构（以及一点运气）来确保顺利。但更常见的是，我所交谈的资深工程师——那些从业时间很长的人——走相反的路。当我问他们是否每天使用 AI 时，他们几乎总会说类似“是的，我用 AI 做单元测试和代码审查”。这几乎总是表明他们不信任 AI 构建应用核心的关键代码，只在不会导致生产缺陷的地方使用 AI。

I think this excerpt from a recent (and excellent) article in Ars Technica, “Cognitive surrender” leads AI users to abandon logical thinking, sums up how many experienced developers feel about working with AI:我认为 Ars Technica 最近（且极佳）的一篇文章《认知投降》让 AI 使用者放弃逻辑思考，概括了许多有经验的开发者对 AI 工作的感受：

When it comes to large language model-powered tools, there are generally two broad categories of users. On one side are those who treat AI as a powerful but sometimes faulty service that needs careful human oversight and review to detect reasoning or factual flaws in responses. On the other side are those who routinely outsource their critical thinking to what they see as an all-knowing machine.在使用大语言模型驱动的工具时，用户大致分为两类。一类把 AI 当作强大但有时会出错的服务，需要细致的人类监督和审查以发现推理或事实错误。另一类则把 AI 当作全知机器，常规地把关键思考外包给它。

I agree that those are two options for dealing with AI. But I also believe that’s a false choice. “Cognitive surrender,” as the research referenced by the article puts it, is not a good outcome. But neither is reviewing every line of code the AI writes, because that’s so effort-intensive that we may as well just write it all ourselves. (And I can almost hear some of you asking, “What so bad about that?”)我同意这两种处理 AI 的方式。但我也认为这是一种错误的二选一。“认知投降”，正如文章引用的研究所说，并不是一个好结果。但逐行审查 AI 编写的每行代码也不是好办法，因为这工作量太大，倒不如自己全部写完。（我几乎能听到有人问：“这有什么不好？”）

This false choice is what really drives a lot of really good, very experienced senior engineers away from AI-driven development today. We see those two options, and they are both terrible. And that’s why I’m writing this article (and the next few in this Radar series) about quality.这种错误的选择正是让许多优秀且经验丰富的高级工程师远离 AI 驱动开发的根本原因。我们看到的两种选项都糟透了，这也是我在本 Radar 系列以及接下来的几篇文章中写质量主题的原因。

Some shocking numbers about AI coding tools关于 AI 编码工具的一些惊人数字

The Quality Playbook is an open-source skill for AI coding tools like GitHub Copilot, Cursor, Claude Code, and Windsurf. You point it at a codebase, and it generates a complete quality engineering infrastructure for that project: test plans traced to requirements, code review protocols, integration tests, and more. More importantly, it brings back quality engineering practices that much of the industry abandoned decades ago, using AI to do a lot of the quality-related work that used to require a dedicated team.Quality Playbook 是一个开源 AI 编码工具技能，适用于 GitHub Copilot、Cursor、Claude Code 和 Windsurf。你把它指向代码库，它会为该项目生成完整的质量工程基础设施：可追溯到需求的测试计划、代码审查协议、集成测试等。更重要的是，它重新引入了行业在数十年前放弃的质量工程实践，利用 AI 完成过去需要专门团队的质量相关工作。

I built the Quality Playbook as part of an experiment in AI-driven development and agentic engineering, building an open-source project called Octobatch and writing about the process in this ongoing Radar series. The playbook emerged directly from that experiment. The ideas behind it are over fifty years old, and they work.我在 AI 驱动开发和代理工程的实验中构建了 Quality Playbook，创建了开源项目 Octobatch，并在本 Radar 系列中记录了过程。该手册直接源自该实验。其背后的理念已有五十多年历史，并且行之有效。

Along the way, I ran into a shocking statistic.在此过程中，我遇到一个令人震惊的统计数据。

We already know that many (most?) developers these days use AI coding tools like GitHub Copilot, Claude Code, Gemini, ChatGPT, and Cursor to write production code. But do we trust the code those tools generate? “Trust in these systems has collapsed to just 33%, a sharp decline from over 70% in 2023.”我们已经知道，如今许多（大多数）开发者使用 GitHub Copilot、Claude Code、Gemini、ChatGPT 和 Cursor 等 AI 编码工具编写生产代码。但我们真的信任这些工具生成的代码吗？“对这些系统的信任已跌至仅 33%，相比 2023 年的 70% 大幅下降。”

That quote is from a Gemini Deep Research report I generated while doing research for this article. 70% dropping to 33%—that sounds like a massive collapse, right?这句话出自我为本文研究而生成的 Gemini 深度研究报告。70% 降至 33%——听起来像是一次巨大的崩塌，对吧？

The thing is, when I checked the sources Gemini referenced, the truth wasn’t nearly as clear-cut. That “over 70% in 2023” number came from a Stack Overflow survey measuring how favorably developers view AI tools. The “33%” number came from a Qodo survey asking whether developers trust the accuracy of AI-generated code. Gemini grabbed both numbers, stripped the context, and stitched them into a single decline narrative. No single study ever measured trust dropping from over 70% to 33%. Which means we’ve got an apples-to-oranges comparison, and it might even technically be accurate (sort of?), but it’s not really the headline-grabber that it seemed to be.问题是，当我检查 Gemini 引用的来源时，事实并没有那么明确。所谓的“2023 年超过 70%”来自 Stack Overflow 调查，衡量开发者对 AI 工具的好感度。而“33%”则来自 Qodo 调查，询问开发者是否信任 AI 生成代码的准确性。Gemini 把这两个数字拼凑在一起，去掉了上下文，制造了一个单一的下降叙事。没有任何单独的研究测量过信任度从 70% 下降到 33%。这意味着我们在进行苹果对橙子的比较，虽然在技术上可能算是准确的（某种程度上），但并不像标题那样引人注目。

So why am I telling you about it?那我为什么要告诉你这个？

Because there are two important lessons from that “shocking” stat. The first is that the overall idea rings true, at least for me. Almost all of us have had the experience of generating code with AI faster than we can verify it, and we ship features before we fully review them.因为这个“惊人”统计背后有两个重要教训。第一点对我而言基本成立：几乎所有人都有 AI 生成代码快于我们验证的经历，而且我们在完全审查之前就已经发布了功能。

The second is that when Gemini created the report, the AI fabricated the most alarming version of the story from real but unrelated data points. If I’d just cited it without checking the sources, there’s a pretty good chance it would get published, and you might even believe it. That’s ironically self-referential, because it’s literally the trust problem the survey is supposedly measuring. The AI produced something that looked authoritative, felt correct, and was wrong in ways that only careful verification could catch. If you want to understand why over 70% of developers don’t fully trust AI-generated code, you just watched it happen.第二点是，当 Gemini 编写报告时，AI 用真实但不相关的数据点捏造了最惊人的版本。如果我仅仅引用而不核实来源，很可能会被发布，甚至让你相信。这具有讽刺意味，因为这正是调查本应衡量的信任问题。AI 生成了看似权威、感觉正确，却只有通过仔细验证才能发现错误的内容。如果你想了解为何超过 70% 的开发者不完全信任 AI 生成的代码，只需亲眼目睹其过程。

One reason many of us don’t trust AI-generated choice is because there’s a growing gap between how fast AI can generate code and how well we can verify that the code actually does what we intended. The usual response to this verification gap is to adopt better testing tools. And there are plenty of them: test stub generators, diff reviewers, spec-first frameworks. These are useful, and they solve real problems. But they generally share a blind spot: they work with what the code does, not with what it’s supposed to do. Luckily, the intent is sitting right there: in the specs, the schemas, the defensive code, the history of the AI chats about the project, even the variable names and filenames. We just need a way to use it.许多人不信任 AI 生成代码的一个原因是，AI 生成代码的速度与我们验证代码是否符合预期的速度之间的差距在不断扩大。对此的常见回应是采用更好的测试工具。相关工具很多：测试桩生成器、差异审查器、规格优先框架。这些工具有用，能解决真实问题，但它们通常有盲点：它们关注代码实际做了什么，而不是它应该做什么。幸运的是，意图就在那儿：规格、模式、容错代码、项目的 AI 对话历史，甚至变量名和文件名。我们只需要一种方法来利用这些信息。

AI-driven development needs its own quality practices, and the discipline we need already exists. It was just (unfairly) considered too expensive to use… until AI made it cheap.AI 驱动开发需要自己的质量实践，而我们已经拥有的学科正好可以满足需求。它曾被（不公平地）认为成本太高……直到 AI 让它变得廉价。

(Re-)introducing quality engineering（重新）引入质量工程

There’s a difference between knowing that code works and knowing that it does what it’s supposed to do. It’s the difference between “does this function return the right value?” and “does this system fulfill its purpose?”—and as it turns out, that’s one of the oldest problems in software engineering. In fact, as I talked about in a previous Radar article, Prompt Engineering Is Requirements Engineering, it was the source of the original “software crisis.”知道代码能运行和知道它能完成预期任务是有区别的。这是“这个函数返回正确值吗？”与“这个系统实现了它的目的吗？”之间的差别——而且这正是软件工程最古老的问题之一。事实上，正如我在之前的 Radar 文章《Prompt Engineering Is Requirements Engineering》中所说，它是最初“软件危机”的根源。

The software crisis was the term people used across our industry back in the 1960s when they were coming to grips with large software projects around the world that were routinely delivered late, over budget, and delivering software that didn’t do what it was supposed to do. At the 1968 NATO Software Engineering Conference—the conference that introduced the term “software engineering”—some of the top experts in the industry talked about how the crisis was caused by the developers and their stakeholders had trouble understanding the problems they were solving, communicating those needs clearly, and making sure that the systems they delivered actually met their users’ needs. Nearly two decades later, Fred Brooks made the same argument in his pioneering essay, No Silver Bullet: no tool can, on its own, eliminate the inherent difficulty of understanding what needs to be built and communicating that intent clearly. And now that we talk to our AI development tools the same way we talk to our teammates, we’re more susceptible than ever to that underlying problem of communication and shared understanding.软件危机是 1960 年代业界用来描述全球大型软件项目经常延期、超预算且交付的产品不符合预期的术语。1968 年 NATO 软件工程会议——首次提出“软件工程”一词的会议——上，行业顶尖专家指出危机源于开发者和利益相关者难以理解所要解决的问题、清晰沟通需求以及确保交付的系统真正满足用户需求。近二十年后，弗雷德·布鲁克斯在其开创性论文《No Silver Bullet》中再次提出相同观点：没有任何工具能够单独消除理解需求和清晰传达意图的固有困难。而如今我们以同伴的方式与 AI 开发工具对话，这让我们比以往更容易陷入沟通和共享理解的根本问题。

An important part of the industry’s response to the software crisis was quality engineering, a discipline built specifically to close the gap between intent and implementation by defining what “correct” means up front, tracing tests back to requirements, and verifying that the delivered system actually does what it’s supposed to do. For years it was standard practice for software engineering teams to include quality engineering phases in all projects. But few teams today do traditional quality engineering. Understanding why it got left behind by so many of us, more importantly, what it can do for us now, can make a huge difference for agentic engineering and AI-driven development today.针对软件危机，业界的一个重要回应是质量工程，这是一门专门用于通过预先定义“正确”含义、将测试追溯到需求并验证交付系统是否符合预期来弥合意图与实现之间差距的学科。多年来，软件工程团队在所有项目中都将质量工程阶段作为标准实践。但如今很少有团队仍然采用传统质量工程。了解它为何被大多数人抛弃，以及它现在能为我们做什么，对今天的代理工程和 AI 驱动开发意义重大。

Starting in the 1950s, three thinkers built the intellectual foundation that manufacturing used to become dramatically more reliable.从 1950 年代起，三位思想家奠定了制造业实现显著可靠性的理论基础。

• W. Edwards Deming argued that quality is built into the process, not inspected in after the fact. He taught us that you don’t test your way to a good product; you design the system that produces it.W. Edwards Deming 主张质量是内嵌在过程中的，而不是事后检查。他教导我们，不能通过测试来获得好产品；必须设计出能产出好产品的系统。
• Joseph Juran defined quality as fitness for use: not just “does it work?” but ”does it do what it’s supposed to do, under real conditions, for the people who actually use it?”Joseph Juran 将质量定义为适用性：不仅要“它能工作吗”，更要“它在真实条件下、对实际使用者而言是否实现了预期功能”。
• Philip Crosby made the business case: quality is free, because building it in costs less than finding and fixing defects after the fact. By the time I joined my first professional software development team in the 1990s, these ideas were standard practice in our industry.Philip Crosby 提出了商业论点：质量是免费的，因为在前期构建质量的成本低于事后发现并修复缺陷的成本。到我在 1990 年代加入第一支专业软件开发团队时，这些理念已成为行业标准。

These ideas revolutionized software quality, and the people who put them into practice were called quality engineers. They built test plans traced to requirements, ran functional testing against specifications, and maintained living documentation that defined what “correct” meant for each part of the system.这些理念彻底改变了软件质量，实践它们的人被称为质量工程师。他们编写可追溯到需求的测试计划，依据规格进行功能测试，并维护活文档，定义每个系统部件的“正确”含义。

So why did all of this disappear from most software teams? (It’s still alive in regulated industries like aerospace, medical devices, and automotive, where traceability is mandated by law, and a few brave holdouts throughout the industry.) It wasn’t because it didn’t work. Quality engineering got cut because it was perceived as expensive. Crosby was right that quality is free: the cost of building it in is far more than made up for by the savings you get from not finding and fixing defects later. But the costs come at the beginning of the project and the savings come at the end. In practice, that means when the team blows a deadline and the manager gets angry and starts looking for something to cut, the testing and QA activities are easy targets because the software already seems to be complete.那么，为什么这些做法在大多数软件团队中消失了？（在航空、医疗设备和汽车等受监管行业仍然存活，且有少数勇敢的坚持者。）并不是因为它们不起作用。质量工程被削减是因为被视为昂贵。Crosby 说得对，质量是免费的：前期投入的成本远低于后期因缺陷导致的节约。但成本在项目初期，而节省在项目后期。实际操作中，这意味着当团队错过截止日期、经理生气并寻找削减项目的地方时，测试和 QA 活动往往成为首选削减目标，因为软件看起来已经完成。

On top of the perceived expense, quality engineering required specialists. Building good requirements, designing test plans, and planning and running functional and regression testing are real, technical skills, and most teams simply didn’t have anyone (or, more specifically, the budget for anyone) who could do those jobs.除了被认为费用高，质量工程还需要专门人才。编写良好需求、设计测试计划以及规划和执行功能和回归测试都是技术性强的技能，而大多数团队根本没有（或者更准确地说，没有预算雇佣）能够胜任这些工作的人。

Quality engineering may have faded from our projects and teams over time, but the industry didn’t just give up on many of its best ideas. Developers are nothing if not resourceful, and we built our own quality practices—three of the most popular are test-driven development, behavior-driven development, agile-style iteration—and these are genuinely good at what they do. TDD keeps code honest by making you write the test before the implementation. BDD was specifically designed to capture requirements in a form that developers, testers, and stakeholders can all read (though in practice, most teams strip away the stakeholder involvement and it devolves into another flavor of integration testing). Agile iteration tightens the feedback loop so you catch problems earlier.质量工程可能在我们的项目和团队中逐渐淡出，但业界并没有放弃其最佳理念。开发者本身足够足智多谋，我们自行构建了自己的质量实践——最流行的三种是测试驱动开发（TDD）、行为驱动开发（BDD）和敏捷迭代，这些在各自领域都相当有效。TDD 通过先写测试再实现代码来保持代码的诚实性。BDD 专为以所有相关方都能阅读的形式捕获需求（尽管在实践中，大多数团队去掉了利益相关者的参与，沦为另一种集成测试）。敏捷迭代则通过紧密的反馈回路让问题更早被发现。

Those newer quality practices are practical and developer-focused, and they’re less expensive to adopt than traditional quality engineering in the short run because they live inside the development cycle. The upside of those practices is that development teams can generally implement them on their own, without asking for permission or requiring experts. The tradeoff, however, is that those practices have limited scope. They verify that the code you’re writing right now works correctly, but they don’t step back and ask whether the system as a whole fulfills its original intent. Quality engineering, on the other hand, establishes the intent of the system before the development cycle even begins, and keeps it up to date and feeds it back to the team as the project progresses. That’s a huge piece of the puzzle that got lost along the way.这些新兴的质量实践更实用、面向开发者，并且在短期内比传统质量工程更易采纳，因为它们嵌入在开发周期中。其优势在于开发团队通常可以自行实现，无需额外授权或专家介入。然而，这些实践的范围有限。它们只能验证当前编写的代码是否正确，却不回头审视整个系统是否实现了最初的意图。相比之下，质量工程在开发周期开始前就确定系统意图，并在项目推进过程中保持更新并反馈给团队。这是整个拼图中被遗失的关键部分。

Those highly effective quality engineering practices got cut from most software engineering teams because they were viewed as expensive, not because they were wrong. When you’re doing AI-driven development, you’re actually running into exactly the same problem that quality engineering was built to solve. You have a “team”—your AI coding tools—and you need a structured process to make sure that team is building what you actually intend. Quality engineering is such a good fit for AI-driven development because it’s the discipline that was specifically designed to close that gap between what you ask for and what gets built.这些高效的质量工程实践之所以被大多数软件工程团队削减，是因为被视为昂贵，而非错误。当你进行 AI 驱动开发时，实际上正面临质量工程试图解决的同样问题。你拥有一个“团队”——你的 AI 编码工具——需要一个结构化的流程来确保该团队构建的正是你真正想要的。质量工程正好适合 AI 驱动开发，因为它专为弥合需求与实现之间的差距而设计。

What nobody expected is that AI would make it cheap enough in the short run to bring quality engineering back to our projects.没人预料到 AI 会让它在短期内变得足够廉价，以至于我们可以把质量工程重新带回项目中。

Introducing the Quality Playbook介绍 Quality Playbook

I’ve long suspected that quality engineering would be a perfect fit for AI-driven development (AIDD), and I finally got a chance to test that hypothesis. As part of my experiment with AIDD and agentic engineering (which I’ve been writing about in The Accidental Orchestrator and the rest of this series), I built the Quality Playbook, a skill for AI tools like Cursor, GitHub Copilot, and Claude Code that lets you bring these highly effective quality practices to any project, using AI to do the work that used to require a dedicated quality engineering team. Like other AI skills and agents, it’s a structured document that plugs into an AI coding agent and teaches it a specific capability. You point it at a codebase, and the AI explores the code, reads whatever specifications and documentation it can find, and generates a complete quality infrastructure tailored to that project. The Quality Playbook is now part of awesome-copilot, a collection of community-contributed agents (and I’ve also opened a pull request to add it to Anthropic’s repository of Claude Code skills).我早就怀疑质量工程与 AI 驱动开发（AIDD）是完美匹配的，终于有机会验证这一假设。作为我在 AIDD 与代理工程实验的一部分（我在《The Accidental Orchestrator》以及本系列其余文章中都有所记录），我构建了 Quality Playbook——一个适用于 Cursor、GitHub Copilot 和 Claude Code 等 AI 工具的技能，让你可以将这些高效的质量实践带入任何项目，利用 AI 完成过去需要专门质量工程团队的工作。和其他 AI 技能和代理一样，它是一个结构化文档，插入 AI 编码代理并教会它特定能力。你把它指向代码库，AI 会探索代码，读取所有可用的规格和文档，并生成针对该项目的完整质量基础设施。Quality Playbook 现已成为 awesome‑copilot 项目的一部分，我也已提交 Pull Request 将其加入 Anthropic 的 Claude Code 技能库。

What does “quality infrastructure” actually mean? Think about what a quality engineering team would build if you hired one. A good quality engineer would start by defining what “correct” means for your project: what the system is supposed to do, grounded in your requirements, your domain, what your users actually need. From there, they’d write tests traced to those requirements, build a code review process that checks whether the code implements what it’s supposed to, design integration tests that verify the whole system works together, and set up an audit process where independent reviewers check the code against its original intent.“质量基础设施”到底指什么？想象如果你雇佣了一个质量工程团队，他们会做什么。优秀的质量工程师会先定义项目的“正确”含义：系统应做什么，基于你的需求、业务领域以及用户真实需求。随后，他们会编写追溯到这些需求的测试，建立检查代码是否实现预期的代码审查流程，设计验证整个系统协同工作的集成测试，并设置独立审查员对照原始意图检查代码的审计流程。

That’s what the playbook generates. Developers using AI tools have been rediscovering the value of requirements, and spec-driven development (SDD) has become very popular. You don’t need to be practicing strict spec-driven development to use it. The playbook infers your project’s intent from whatever artifacts are available: chat logs, schemas, README files, code comments, and even defensive code patterns. If you have formal specs, great; if not, the AI pieces together what “correct” means from the evidence it can find.这就是手册生成的内容。使用 AI 工具的开发者已经重新发现需求的重要性，规格驱动开发（SDD）也变得非常流行。你不必严格遵循 SDD 才能使用它。手册会从可用的任何制品中推断项目意图：聊天记录、模式、README、代码注释，甚至防御性代码模式。如果你有正式规格，那很好；如果没有，AI 会根据能找到的证据拼凑出“正确”意味着什么。

Once the playbook figures out the intent of the code, it creates quality infrastructure for the project. Specifically, it generates ten deliverables:一旦手册弄清代码意图，它就会为项目创建质量基础设施。具体来说，它会生成十个交付物：

• Exploration and requirements elicitation (EXPLORATION.md): Before the playbook writes anything, it spends an entire phase reading the code, documentation, specs, and schemas, and writes a structured exploration document that maps the project’s architecture and domain. The most common failure mode in AI-generated quality work is producing generic content that could apply to any project. The exploration phase forces the AI to ground everything in this specific codebase, and serves as an audit trail: if the requirements end up wrong, you can trace the problem back to what the exploration discovered or missed.探索与需求获取（EXPLORATION.md）：在手册写任何东西之前，它会花整整一个阶段阅读代码、文档、规格和模式，并编写结构化的探索文档，映射项目的架构和业务领域。AI 生成质量工作中最常见的失败模式是产生可以套用到任何项目的通用内容。探索阶段迫使 AI 将一切基于特定代码库，并作为审计轨迹：如果需求出错，你可以追溯到探索阶段的发现或遗漏。
• Testable requirements (REQUIREMENTS.md): The most important deliverable. Building on the exploration, a five-phase pipeline extracts the actual intent of the project from code, documentation, AI chats, messages, support tickets, and any other project artifacts you can give it. The result is a specification document that a new team member or AI agent can read top-to-bottom and understand the software. Each requirement is tagged with an authority tier and linked to use cases that become the connective tissue tying requirements to integration tests to bug reports.可测试需求（REQUIREMENTS.md）：最重要的交付物。基于探索阶段，五阶段流水线从代码、文档、AI 对话、消息、支持工单以及任何其他项目制品中提取实际意图。结果是一个规格文档，新成员或 AI 代理可以从头到尾阅读并理解软件。每个需求都有权威层级标签，并关联用例，形成需求、集成测试和缺陷报告之间的纽带。
• Quality constitution (QUALITY.md): Defines what “correct” means for your specific project, grounded in your actual domain. Every standard has a rationale explaining why it matters, because without the rationale, a future AI session will argue the standard down.质量宪章（QUALITY.md）：为你的特定项目定义“正确”含义，基于实际业务领域。每条标准都有理由说明其重要性，因为没有理由，未来的 AI 会质疑该标准。
• Spec-traced functional tests: Tests generated from the requirements, not from source code. That difference matters: a test generated from source code verifies that the code does what the code does, while a test traced to a spec verifies that the code does what you intended.规格追溯功能测试：从需求而非源代码生成的测试。区别在于：基于源代码的测试验证代码做了什么，而规格追溯的测试验证代码是否实现了你的意图。
• Three-pass code review protocol with bug reports and regression tests: Three mandatory review passes, each using a different lens: structural review with anti-hallucination guardrails, requirement verification (where you catch things the code doesn’t do that it was supposed to), and cross-requirement consistency checking. Every confirmed bug gets a regression test and a patch file.三遍代码审查协议，附缺陷报告和回归测试：三次强制审查，每次使用不同视角：结构审查（防幻觉护栏）、需求验证（捕获代码未实现的需求）以及跨需求一致性检查。每个确认的缺陷都会生成回归测试和补丁文件。
• Consolidated bug report (BUGS.md): Every confirmed bug with full reproduction details, severity calibrated to real-world impact, and a spec basis citing the specific documentation the code violates. Maintainers respond differently to ”your code violates section X.Y of your own spec” than to ”this looks like it might be a bug.”合并缺陷报告（BUGS.md）：每个确认的缺陷都包含完整的复现细节、依据真实影响的严重性评级，以及引用具体违规规格的依据。维护者对“你的代码违反了你自己规格的 X.Y 节”与“这看起来像是缺陷”的响应不同。
• TDD red/green verification: For each confirmed bug, a regression test runs against unpatched code (must fail), then the fix is applied and the test reruns (must pass). When you tell a maintainer ”here’s a test that fails on your current code and passes with this one-line fix,” that’s qualitatively different from a bug report.TDD 红/绿验证：对每个确认的缺陷，先在未修补代码上运行回归测试（必须失败），再应用修复并重新运行测试（必须通过）。当你告诉维护者“这里有一个在当前代码上失败、在此修复后通过的测试”时，这与普通缺陷报告的性质截然不同。
• Integration test protocol: A structured test matrix that an AI agent can pick up and execute autonomously, without asking clarifying questions. Every test specifies the exact command, what it proves, and specific pass/fail criteria. Field names and types are read from actual source files, not recalled from memory, as an anti-hallucination mechanism.集成测试协议：结构化的测试矩阵，AI 代理可以自行获取并执行，无需澄清问题。每个测试明确命令、验证内容以及通过/失败标准。字段名和类型直接读取自实际源文件，而非记忆中的内容，以防幻觉。
• Council of Three multi-model spec audit: Three independent AI models audit the codebase against the requirements. The triage uses confidence weighting, not majority vote: findings from all three are near-certain, two are high-confidence, and findings from only one get a verification probe rather than being dismissed. The most valuable findings are often the ones only one model catches.三模型规格审计委员会：三种独立 AI 模型审计代码库与需求的符合度。分级使用置信度加权，而非多数投票：三者都发现的结论为“几乎确定”，两者发现为“高置信”，仅一者发现则进行验证探测而非直接驳回。最有价值的发现往往是仅被单一模型捕获的。
• AGENTS.md bootstrap file: A context file that future AI sessions read first, so they inherit the full quality infrastructure. Without it, every new session starts from zero. With it, the quality constitution, requirements, and review protocols carry forward automatically across every session that touches the codebase.AGENTS.md 引导文件：后续 AI 会话首先读取的上下文文件，使其继承完整的质量基础设施。没有它，每次新会话都从零开始。有了它，质量宪章、需求和审查协议会自动在每次触及代码库的会话中传递。

The third option第三种选择

I started this article by talking about a false choice: either we surrender our judgment to the AI, or get stuck reviewing every line of code it writes. The reality is much more nuanced, and, in my opinion, a lot more interesting, if we have a trustworthy way to verify that the code we worked with the AI to build actually does what we intended. It’s not a coincidence that this is one of the oldest problems in software engineering, and not surprising that AI can help us with it.我在文章开头谈到的错误选择是：要么把判断交给 AI，要么陷入逐行审查的泥潭。现实要复杂得多，也更有趣，只要我们拥有可信赖的方式来验证与 AI 合作构建的代码是否实现了我们的意图。这并非巧合，它是软件工程最古老的问题之一，AI 正好可以帮助我们解决。

The Quality Playbook leans heavily on classic quality engineering techniques to do that verification. Those techniques work very well, and that gives us the more nuanced option: using AI to help us write our code, and then using it to help us trust what it built.Quality Playbook 大量借鉴经典质量工程技术来完成验证。这些技术效果显著，为我们提供了更细致的选项：使用 AI 帮助编写代码，然后再利用 AI 帮助我们信任它所构建的内容。

That’s not a gimmick or a paradox. It works because verification is exactly the kind of structured, specification-driven work that AI is good at. Writing tests traced to requirements, reviewing code against intent, checking that the system does what it’s supposed to do under real conditions. These are the things quality engineers used to do across the whole industry (and still do in the highly regulated parts of it). They’re also things that AI can do well, as long as we tell it what “correct” means.这不是噱头或悖论。它之所以有效，是因为验证正是 AI 擅长的结构化、规格驱动工作。编写追溯到需求的测试、根据意图审查代码、检查系统在真实条件下是否按预期工作——这些都是质量工程师在整个行业（尤其是受监管领域）长期从事的工作，也是 AI 在我们明确“正确”含义后能够胜任的任务。

The experienced engineers I talked about at the beginning of this article, the ones who only use AI for unit tests and code reviews, aren’t wrong to be cautious. They’re right that we can’t just trust whatever output the AI spits out. But limiting AI to just the “safe” parts of our projects keeps us from taking advantage of such an important set of tools. The way out of this quagmire is to build the infrastructure that makes the rest of it trustworthy too. Quality engineering gives us that infrastructure, and AI makes it cheap enough to actually use on all of our projects every day.文章开头提到的那些只在单元测试和代码审查中使用 AI 的资深工程师并非没有道理。他们对 AI 输出保持警惕是对的。但把 AI 限制在项目的“安全”部分，会让我们错失重要工具的全部价值。走出困境的办法是构建让其余部分也可信赖的基础设施。质量工程提供了这种基础设施，而 AI 让它足够廉价，能够在我们所有项目中每日使用。

In the next few articles, I’ll show you what happened when I pointed the Quality Playbook at real, mature open-source codebases and it started finding real bugs, how the playbook emerged from my AI-driven development experiment, what the quality engineering mindset looks like in practice, and how we can learn important lessons from that experience that apply to all of our projects.在接下来的几篇文章中，我将展示将 Quality Playbook 指向真实、成熟的开源代码库时它是如何发现真实缺陷的，手册是如何从我的 AI 驱动开发实验中诞生的，质量工程思维在实践中是什么样子，以及我们可以从这些经验中学到哪些对所有项目都有价值的教训。

The Quality Playbook is open source and works with GitHub Copilot, Cursor, and Claude Code. It’s also available as part of awesome-copilot. You can try it out today by downloading it into your project and asking the AI to generate the quality playbook. The whole process takes about 10-15 minutes for a typical codebase. I’ll cover more details on running it in future articles in this series.Quality Playbook 是开源的，兼容 GitHub Copilot、Cursor 和 Claude Code。它也作为 awesome‑copilot 的一部分提供。你可以今天就下载到项目中，让 AI 生成质量手册。对典型代码库来说，整个过程大约需要 10‑15 分钟。我将在本系列的后续文章中详细介绍运行步骤。

Disclosure: Aspects of the methodology described in this article are the subject of US Provisional Patent Application No. 64/044,178, filed April 20, 2026 by the author. The open-source Quality Playbook project (Apache 2.0) includes a patent grant to users of that project under the terms of the Apache 2.0 license.披露：本文所述方法的部分内容已于 2026 年 4 月 20 日以美国临时专利申请号 64/044,178 提交。开源 Quality Playbook 项目（Apache 2.0）在 Apache 2.0 许可证条款下向项目用户授予专利许可。