Five studies changing how I think about AI in software engineering五项改变我软件工程 AI 认知的研究
AI compressed the upstream work. What does that mean for everything downstream?AI 压缩了上游工作，这对下游环节意味着什么？
Welcome to the latest issue of Engineering Enablement, a weekly newsletter sharing research and perspectives on developer productivity.欢迎阅读本期《工程赋能》（Engineering Enablement），这是一份每周分享开发者生产力研究与观点的通讯。
🗓 Join me on July 23 for a readout of the upcoming State of AI Impact in Engineering: Q2 Report. We’ll discuss new findings from DX’s data on AI tool usage, spend, and impact across 500+ organizations. Register here.🗓 欢迎参加 7 月 23 日的线上分享会，届时我将解读即将发布的《工程领域 AI 影响现状：第二季度报告》。我们将探讨 DX 基于 500 多家机构的 AI 工具使用、投入及影响数据所发现的新结论。点击此处注册。
Every once in a while, several independent papers arrive at roughly the same time and collectively tell a bigger story than any one of them does alone. This week, I’m sharing five recent papers that have significantly influenced how I’m thinking about AI and software engineering.有时，多篇独立论文会几乎同时出现，它们共同讲述了一个比单篇论文更宏大的故事。本周，我将分享五篇近期对我关于 AI 与软件工程的思考产生重大影响的论文。
Each paper tackles a different question. Some measure the productivity impact of AI coding assistants. Others examine how those gains propagate through the software delivery process, explore what developers actually want from future AI systems, or reconsider the kinds of debt we should be paying attention to in an AI-assisted world.每篇论文探讨的问题各不相同。有的衡量了 AI 编程助手对生产力的影响，有的研究了这些收益如何渗透到软件交付流程中，有的探讨了开发者对未来 AI 系统的真实诉求，还有的重新审视了在 AI 辅助时代我们应该关注哪些债务。
Despite coming from different research groups and using very different methodologies, they all seem to be converging on the same underlying story.尽管这些研究来自不同的团队，使用的方法论也迥异，但它们似乎正汇聚于同一个核心叙事。
AI is compressing the upstream work of software engineering. The more I sat with these papers, the less I found myself asking, “Is AI making developers faster?” and the more I found myself asking, “What happens after the code is written?” Are we actually shipping more value? Where do the new bottlenecks emerge? And what are the costs if understanding can’t keep pace with generation?AI 正在压缩软件工程的上游工作。随着我对这些论文的深入思考，我不再纠结于“AI 是否让开发者变快了？”这个问题，而是更多地思考：“代码写完之后会发生什么？”我们真的交付了更多价值吗？新的瓶颈出现在哪里？如果理解力跟不上生成速度，我们又将付出怎样的代价？
After reading these five papers, I came away with one overarching conclusion: we’re generating code faster than we’re generating the systems needed to safely understand, verify, and deliver it.读完这五篇论文，我得出了一个核心结论：我们生成代码的速度，远超我们构建安全理解、验证和交付这些代码所需系统的速度。
A quick note on disclosure: three of these papers come from people I know and work with extensively. None of the papers are mine.披露说明：其中三篇论文的作者是我熟识并有深度合作的伙伴。这些论文均非我所写。
Here they are, in the order I’d recommend reading them.以下是按建议阅读顺序排列的论文列表。
1. GitHub Copilot and Developer Productivity1. GitHub Copilot 与开发者生产力
Paper: Heilman, A., Kyllo, A., Murphy-Hill, E. GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis.论文：Heilman, A., Kyllo, A., Murphy-Hill, E. 《GitHub Copilot 与开发者生产力：一项观察性剂量反应分析》。
The first paper I want to highlight tackles the familiar question of whether GitHub Copilot makes developers more productive, but it does so with one of the more clever research designs I’ve seen.我想介绍的第一篇论文探讨了“GitHub Copilot 是否提高了开发者生产力”这一经典问题，但它采用了我所见过的最巧妙的研究设计之一。
Rather than simply comparing Copilot users to non-users (which are getting harder and harder to find), the authors control for Active Coding Time (i.e., how much time developers spend actively engaging with development tools) and examine how productivity changes within the same engineer over 43 weeks across a population of 16,223 developers.作者没有简单地对比 Copilot 用户与非用户（因为非用户群体正变得越来越难找），而是引入了“活跃编码时间”（即开发者投入在开发工具上的实际时间）作为控制变量，并追踪了 16,223 名开发者在 43 周内的生产力变化。
The payoff of this design is that it compares engineers to themselves rather than to one another. Using that approach, the authors found that weeks with the highest Copilot usage were associated with ~40% more completed PRs per hour of coding time than weeks with no usage.这种设计的优势在于，它将工程师与自身进行纵向对比，而非横向对比。研究发现，在 Copilot 使用率最高的几周，每小时编码时间的 PR 完成量比不使用时高出约 40%。
The relationship showed a clear dose-response pattern (a way to do a causal analysis, once everyone is already using the tools). More Copilot engagement was associated with more PR throughput, although the gains appeared to level off at very high usage.这种关系呈现出清晰的“剂量反应”模式（这是一种在普及工具后进行因果分析的方法）。Copilot 使用越多，PR 的吞吐量就越高，尽管在极高使用频率下，收益增长趋于平缓。
The authors ran seven robustness and falsification tests to rule out alternative explanations (team-level effects, generic AI engagement, PR slicing, shifts toward easier work). The positive association remained remarkably consistent.作者进行了七项稳健性和伪造性测试，排除了其他可能的解释（如团队效应、通用 AI 参与度、PR 拆分、工作内容简化等）。这种正相关性依然保持得非常稳健。
Interestingly, the gains were not concentrated in tiny PRs. The strongest effects were observed for larger PRs (7+ files), arguing against the idea that developers are simply breaking work into smaller units.有趣的是，这些收益并非集中在微小的 PR 上。研究发现，大型 PR（7 个以上文件）的收益最显著，这反驳了“开发者只是在把工作拆得更细”的观点。
It’s a thoughtful analysis and shows that we’re not just coding more, we’re increasing coding efficiency as well. These findings anchor many of the studies that follow in this roundup.这是一项严谨的分析，它表明我们不仅是在写更多代码，还在提升编码效率。这些发现为后续的各项研究奠定了基础。
2. Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools2. 编写代码与交付代码：不同代际 AI 编程工具的生产力影响
Paper: Demirer, M., Musolff, L., Yang, L. Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools.论文：Demirer, M., Musolff, L., Yang, L. 《编写代码与交付代码：不同代际 AI 编程工具的生产力影响》。
The next paper I’m highlighting was published by the National Bureau of Economic Research. It analyzes AI adoption across 100,000+ GitHub developers and asks a more nuanced question than Heilman’s: when AI makes individual coding steps faster, how much of that gain actually survives all the way to shipped software?我想介绍的下一篇论文由美国国家经济研究局（NBER）发布。它分析了 10 万多名 GitHub 开发者的 AI 采纳情况，并提出了一个比 Heilman 的研究更细腻的问题：当 AI 加速了单个编码步骤时，这些收益中有多少能最终转化为已交付的软件？
The authors examine how AI productivity gains propagate through a hierarchy of software development: lines of code → files → commits → pull requests → projects/repos → releases.作者考察了 AI 生产力收益如何沿着软件开发层级传递：代码行 → 文件 → 提交 → Pull Request → 项目/仓库 → 版本发布。
They found that AI is clearly increasing coding activity, and the gains grow with each generation of tools. They estimate roughly +40% more commits from autocomplete, growing to +140% from interactive coding agents, and finally +180% from autonomous agents.他们发现 AI 确实显著提升了编码活动，且收益随工具代际演进而增长。据估计，自动补全带来了约 40% 的提交增长，交互式编码代理提升至 140%，而自主代理则高达 180%。
However, those gains fall off significantly as work moves through the software delivery process. The largest effects are seen in code generation, but smaller effects appear in repos touched, small still in releases shipped, and ultimately software consumed by users. Even with very large increases in coding activity, the effect on shipped software is much smaller, topping out at roughly +30% more releases. This is illustrated in figure 2 below.然而，随着工作流向软件交付的后续环节，这些收益显著衰减。代码生成环节的效应最为明显，但在涉及的仓库数量、发布版本数量以及最终用户消费的软件上，影响逐渐减弱。即便编码活动大幅增加，对最终交付软件的影响也小得多，最高仅增长约 30%。如下图 2 所示。
One of the findings I found most interesting is that they estimate a low elasticity of substitution (~0.25) between AI-generated output and human effort. That’s an economics concept that measures how replaceable human work is with AI output. As a methodology nerd, and someone with an economics degree, I found this particularly clever — they infer this elasticity from how AI productivity gains attenuate across the delivery process. Their estimate suggests AI and human work are still largely complements rather than substitutes, with substantial human effort still required to review, integrate, validate, and ship software.最令我感兴趣的发现之一是，他们估算出 AI 生成输出与人类努力之间的替代弹性较低（约 0.25）。这是一个经济学概念，用于衡量 AI 输出对人类劳动的可替代性。作为一名方法论爱好者和经济学学位持有者，我觉得这非常精妙——他们通过 AI 生产力收益在交付过程中的衰减程度推导出了这一弹性。他们的估算表明，AI 和人类工作在很大程度上仍是互补而非替代关系，软件的审查、集成、验证和交付依然需要大量的人力投入。
One open question is whether the observed fall-off through the delivery process is some fundamental limit of software engineering, or simply the fact that organizations have not yet adapted their processes to an agentic world.一个悬而未决的问题是：这种在交付过程中的收益衰减是软件工程的根本局限，还是仅仅因为组织尚未调整流程以适应 AI 代理时代？
If Heilman tells you Copilot is making engineers measurably faster, this paper asks the harder question: faster at what, exactly?如果说 Heilman 的研究告诉你 Copilot 让工程师变快了，那么这篇论文则提出了一个更深刻的问题：到底在什么环节变快了？
3. The Impact of AI Coding Assistants on Software Engineering3. AI 编程助手对软件工程的影响
Paper: Vella, A., Blincoe, K. The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study.论文：Vella, A., Blincoe, K. 《AI 编程助手对软件工程的影响：一项纵向研究》。
The next study I want to highlight is unique because it isn’t just a snapshot in time, it’s a six-month longitudinal study of 95 professional software engineers. It also calls into question a relationship that we’ve long believed to be a bedrock of developer experience.接下来的这项研究非常独特，因为它不是一个时间切片，而是对 95 名专业软件工程师进行的为期六个月的纵向研究。它还质疑了一个我们长期以来视为开发者体验（DevEx）基石的关系。
The study was done using two questionnaires six months apart, mixed-methods, with reflexive thematic analysis on the open-ended responses.该研究通过间隔六个月的两次问卷调查进行，采用了混合研究方法，并对开放式回答进行了反思性主题分析。
Vella found that productivity perceptions were stable and strongly positive over time. 84% of study participants reported improvement at both time points. Consistent with the first two studies in this round-up, the story of accelerated throughput is real and persistent.Vella 发现，开发者对生产力的感知在六个月内保持稳定且高度积极。84% 的参与者在两个时间点都报告了效率提升。这与本综述中的前两项研究一致，即“吞吐量加速”是一个真实且持续的现象。
The really striking finding is what the authors call the productivity-experience paradox. Among the matched cohort, the share of engineers reporting worse DevEx on at least one dimension nearly doubled in just six months, from 14% to 27%. Flow state was the most vulnerable; cognitive load eroded modestly; feedback loops actually improved.真正令人震惊的发现是作者所谓的“生产力与体验悖论”。在匹配的队列中，认为开发者体验（DevEx）在至少一个维度上变差的工程师比例在短短六个月内几乎翻了一番，从 14% 升至 27%。其中“心流状态”最易受损，认知负荷略有增加，而反馈回路确实得到了改善。
More importantly: while the cross-sectional correlations between DevEx and productivity were strong, the change scores didn’t correlate. Productivity and developer experience appear to be decoupling over time in AI-assisted workflows. For those of us who’ve spent years working with the SPACE and DevEx frameworks, that’s worth sitting with.更重要的是：虽然 DevEx 与生产力之间的横向相关性很强，但它们的变化分数并不相关。在 AI 辅助的工作流中，生产力和开发者体验似乎正在脱钩。对于多年来一直使用 SPACE 和 DevEx 框架的我们来说，这一点值得深思。
While this study didn’t have a particularly large population, the findings were significant and rigorously validated, proving that a study doesn’t have to be massive if the strength of results is strong enough. This longitudinal design is rare and valuable, and the productivity-experience decoupling is the kind of finding worth replicating in larger populations.尽管该研究样本量不算特别大，但其发现具有显著性且经过了严格验证，证明了只要结果足够有力，研究不必非得追求庞大。这种纵向设计非常难得且有价值，“生产力与体验脱钩”这一结论非常值得在更大规模的群体中进行复现。
4. To Copilot and Beyond: 22 AI Systems Developers Want Built4. 迈向 Copilot 之外：开发者希望构建的 22 种 AI 系统
Paper: Choudhuri, R., Badea, C., Bird, C., Butler, J., DeLine, R., Houck, B. AI Where It Matters: Where, Why, and How Developers Want AI Support in Daily Work.论文：Choudhuri, R., Badea, C., Bird, C., Butler, J., DeLine, R., Houck, B. 《AI 在关键之处：开发者在日常工作中希望 AI 提供支持的领域、原因与方式》。
Last year I published a paper called AI Where It Matters, and my co-authors ended up writing a 2nd paper based on the original survey responses (860 Microsoft developers across roles, domains, and geographies). The paper outlines a roadmap of 22 AI tools that developers want beyond just code generation, centered around a concept they call “bounded delegation.” A lot of this echoes what the rest of this round-up is circling:去年我发表了一篇题为《AI 在关键之处》的论文，我的合著者基于原始调查回复（涵盖不同角色、领域和地区的 860 名微软开发者）撰写了第二篇论文。该论文勾勒了开发者除代码生成外所期望的 22 种 AI 工具路线图，其核心概念被称为“有界授权”（bounded delegation）。其中许多观点与本综述的其他内容不谋而合：
The “right-shift” burden. Because AI is speeding up code generation, it’s creating a massive bottleneck downstream. Devs are getting flooded with more code to review, more production incidents to debug, and documentation that falls behind faster than ever.“右移”负担。由于 AI 加速了代码生成，它正在下游制造巨大的瓶颈。开发者被淹没在更多的代码审查、更多的生产事故调试以及更新速度赶不上代码变更的文档中。
The move to verification. Developers don’t want more code-generation assistants; they want AI embedded into verification tasks — tools that automatically assemble log/trace “case files” for on-call incidents, PR reviewers that catch complex business logic flaws before human review, change-aware test generation that knows which assertions actually matter.转向验证。开发者不想要更多的代码生成助手；他们希望将 AI 嵌入到验证任务中——例如自动收集日志/追踪记录以生成事故“案例文件”的工具、能在人工审查前捕获复杂业务逻辑缺陷的 PR 审查器、以及能识别哪些断言真正重要的变更感知测试生成工具。
“Bounded delegation.” There is a strict boundary around where developers want AI to stop. Developers want AI to absorb the tedious “assembly work” surrounding their craft (updating docs, writing edge-case unit tests), but never the core logic, architecture, or critical decision-making. Notably, developers drew this line even for tasks they acknowledged AI could plausibly handle — suggesting it’s not just about capability gaps and won’t move just because models improve.“有界授权”。开发者对 AI 的介入设定了严格的边界。他们希望 AI 处理繁琐的“组装工作”（如更新文档、编写边缘情况的单元测试），但绝不希望 AI 触碰核心逻辑、架构或关键决策。值得注意的是，即便开发者承认 AI 有能力处理某些任务，他们依然划定了这条界限——这表明这不仅仅是能力差距的问题，也不会因为模型性能的提升而消失。
Four non-negotiable guardrails. For future AI tools to be adopted, developers say they must enforce explicit authority scoping (no auto-approvals), clear data provenance, explicit uncertainty signaling (the AI must admit when it doesn’t know something), and least-privilege security access.四项不可妥协的护栏。开发者认为，未来 AI 工具若要被采纳，必须强制执行明确的权限范围（禁止自动审批）、清晰的数据来源、明确的不确定性信号（AI 必须承认自己不知道什么），以及最小权限的安全访问原则。
You can check out both papers and an interactive website here: aka.ms/ai-where-it-matters你可以在此处查看这两篇论文及互动网站：aka.ms/ai-where-it-matters
5. From Technical Debt to Cognitive and Intent Debt5. 从技术债务到认知与意图债务
Paper: Storey, M. From Technical Debt to Cognitive and Intent Debt: Rethinking software health in the age of AI论文：Storey, M. 《从技术债务到认知与意图债务：在 AI 时代重新思考软件健康》。
I’ve saved this for last because I think this is the most important paper I’ve read in a long time. Margaret-Anne Storey makes a generational argument: the metaphor we’ve used for decades to think about software health—technical debt—is no longer sufficient. AI is reducing technical debt (through refactoring, test generation, automated review) while quietly accelerating the accumulation of two other forms of debt that matter more in this era.我把这一篇放在最后，因为我认为这是我近期读过的最重要的论文。Margaret-Anne Storey 提出了一个划时代的观点：我们几十年来用于思考软件健康的隐喻——“技术债务”——已经不够用了。AI 正在减少技术债务（通过重构、测试生成、自动化审查），同时却在悄悄加速积累在这个时代更为关键的另外两种债务。
Technical debt lives in code. It accumulates when implementation decisions compromise future changeability. AI is genuinely helping here.技术债务存在于代码中。当实现决策损害了未来的可变性时，它就会积累。AI 在这方面确实提供了帮助。
Cognitive debt lives in people. It accumulates when a team’s shared understanding of a system erodes faster than it’s replenished. When AI generates the code, developers may accept it without building the same mental model they would have built by writing it themselves. Multiply that across a team and over time, and you get “an accumulation of not knowing.”认知债务存在于人的头脑中。当团队对系统的共识理解流失速度超过其补充速度时，它就会积累。当 AI 生成代码时，开发者可能在没有建立起自己亲手编写代码时所形成的心理模型的情况下就接受了代码。在团队中长期叠加这种效应，就会导致“无知感的积累”。
Intent debt lives in artifacts. It accumulates when the goals, constraints, and rationale that guide a system—the things both humans and AI agents need to work safely—are unclear, unwritten, or forgotten. As more development is AI-assisted, intent debt becomes a first-order constraint on what AI can actually do for you.意图债务存在于工件中。当指导系统的目标、约束和基本原理——即人类和 AI 代理安全工作所需的信息——不清晰、未记录或被遗忘时，它就会积累。随着越来越多的开发工作由 AI 辅助，意图债务成为了 AI 能为你做什么的一项首要制约因素。
The three debts interact and compound. Intent debt causes cognitive debt; cognitive debt causes technical debt; technical debt amplifies cognitive debt. Managing software system health requires attention to all three layers, not just the one easiest to measure.这三种债务相互作用并复合。意图债务导致认知债务；认知债务导致技术债务；技术债务放大认知债务。管理软件系统健康需要关注所有三个层面，而不仅仅是最容易衡量的那一个。
The four practical implications Storey draws are worth reading in full, but the headline is: treat understanding as a deliverable. Just as working code is a product of software development, shared understanding should be treated as a first-class deliverable, not something that happens as a side effect of writing code.Storey 提出的四个实践启示值得通读，但核心要点是：将“理解”视为一种交付成果。正如可运行的代码是软件开发的产物一样，共识理解也应被视为一等交付成果，而不是编写代码的副产品。
Final thoughts结语
Read together, these five papers say something stronger than any of them say individually. AI is genuinely making code generation faster, and the per-engineer efficiency gains are real (Heilman). But those gains don’t survive the trip to shipped software at anywhere near the same magnitude (Demirer). The bottleneck has moved downstream to review, integration, verification, and understanding. Developers feel it, they’re explicitly asking for tools to address those bottlenecks while refusing to delegate the parts of the job they consider craft (Choudhuri). The lived experience of working this way is more uneven than the productivity numbers suggest, with flow and cognitive load eroding even as throughput holds (Vella). And the deepest cost may be one we don’t yet measure: the slow erosion of shared understanding, which is what makes any system safe to change (Storey).综合来看，这五篇论文传达出的信息比单篇论文深刻得多。AI 确实加快了代码生成速度，工程师个人的效率提升也是真实的（Heilman）。但这些收益在转化为已交付软件的过程中，其幅度大打折扣（Demirer）。瓶颈已经转移到了审查、集成、验证和理解环节。开发者感受到了这一点，他们明确要求提供工具来解决这些瓶颈，同时拒绝将他们视为“手艺”的工作交给 AI（Choudhuri）。这种工作方式的实际体验比生产力数据所显示的更为复杂，即便吞吐量保持不变，心流和认知负荷也在流失（Vella）。而最深层的代价可能是我们尚未衡量的：对共识理解的缓慢侵蚀，而这正是任何系统能够安全变更的基础（Storey）。
The bottleneck has moved. Our tools, metrics, and team designs haven’t moved with it yet. That’s where the next several years of work in our field are going to happen.瓶颈已经转移。我们的工具、指标和团队设计尚未跟上这一变化。这正是我们领域未来几年工作的重点所在。
That’s it for this week. And make sure to sign up for my upcoming live research readout covering new findings on AI’s impact, where we’ll discuss data from both DX and the broader industry.本周就到这里。记得报名参加我即将举行的关于 AI 影响的现场研究解读会，届时我们将讨论来自 DX 和更广泛行业的最新数据。
-Brian-Brian
