Building self-improving tax agents with Codex使用 Codex 构建自我改进的税务智能体

How Thrive Holdings and OpenAI co-developed Tax AI for Crete accountants by fusing practitioner expertise with a Codex-driven loopThrive Holdings 与 OpenAI 如何通过将从业者专业知识与 Codex 驱动的循环相结合，共同为 Crete 会计师开发 Tax AI

Real-world systems behave differently in production than they do in a lab, breaking in ways that are hard to anticipate before deployment. Teams often discover those failures after launch, then spend weeks inspecting edge cases, adjusting prompts, and translating production feedback into durable product improvements. The feedback loop is manual and slow, and only improves when an engineer advances it. But today, with thoughtfully designed eval infrastructure, direct access to practitioners and real world environments, and the frontier agentic capabilities of Codex, you can build agents that self-improve.真实世界的系统在生产环境中的表现与实验室不同，会以部署前难以预料的方式出现故障。团队常常在上线后才发现这些故障，然后花费数周时间检查边缘情况、调整提示词，并将生产反馈转化为持久的产品改进。反馈循环是手动且缓慢的，只有在工程师推动时才会进步。但如今，借助精心设计的评估基础设施、直接接触从业者和真实世界环境的渠道，以及 Codex 的前沿智能体能力，你可以构建能够自我改进的智能体。

In this post, we’ll unpack how we used Codex to build this type of agent. Over the past six months, OpenAI forward deployed engineers and researchers along with Thrive Holdings’ engineers collaborated to build Tax AI alongside and for Crete⁠(opens in a new window)’s network of 30+ accounting firms to help prepare increasingly complex tax returns. Instead of relying on engineers to find and fix each failure, Tax AI uses Codex to turn production use into structured signals that fuel autonomous improvement.在这篇文章中，我们将详细剖析如何使用 Codex 构建这类智能体。过去六个月，OpenAI 派驻的工程师和研究人员与 Thrive Holdings 的工程师合作，为 Crete（在新窗口中打开）旗下 30 多家会计师事务所构建 Tax AI，以帮助处理日益复杂的纳税申报。Tax AI 不再依赖工程师去发现和修复每一个故障，而是利用 Codex 将生产使用转化为结构化信号，从而驱动自主改进。

Crete practitioners prepare tens of thousands of tax returns each season which requires working through millions of underlying documents. For medium- to large-complexity filings, data entry alone can take eight hours per return, often involving messy data sources, prior-year documents, and manual extraction and calculation. They pointed us to tax preparation as a significant bottleneck during the busiest stretch of tax season.Crete 的从业者每个季度要准备数万份纳税申报表，需要处理数百万份底层文件。对于中等到高复杂度的申报，仅数据录入每项就可能花费八个小时，通常涉及混乱的数据源、往年文件以及手动提取和计算。他们指出，在报税季最繁忙的时期，税务准备工作是一个重大的瓶颈。

To solve this problem, Tax AI processed 7,000 tax returns across the Crete firms that participated in the pilot this tax season. The system automates much of the time-intensive process of preparing 1040 and 1041 tax returns, but even more compelling than the efficiency gains is that the system itself is measurably better than the version that was first deployed three months ago.为了解决这个问题，Tax AI 在本报税季处理了参与试点的 Crete 公司旗下 7,000 份纳税申报表。该系统将准备 1040 和 1041 纳税申报表这一耗时的流程实现了大量自动化，但比效率提升更具说服力的是，该系统本身比三个月前首次部署的版本有了可衡量的改进。

In Tax AI, practitioners upload source files along with any client-specific notes. Tax AI then creates a tax engine submission, ready for review. It saves practitioners about a third of their time on tax preparation, drafts returns with up to 97% accuracy, and increases throughput by about 50%, creating more room for them to spend time with clients. 在 Tax AI 中，从业者上传源文件以及任何客户特定备注。随后，Tax AI 会生成一份供审核的税务引擎提交内容。它为从业者节省了约三分之一的税务准备时间，起草申报表的准确率高达 97%，并将处理量提高了约 50%，为他们腾出更多时间与客户相处。

We can quantify this improvement by understanding how accurately Tax AI can complete a return without needing correction later. We measure accuracy by checking what share of returns reach 75%, 90%, or 100% correct field completion. At launch, only a quarter of returns were at 75% correct field completion, but within six weeks, 86% hit that mark. The system showed even faster growth at the 90% and 100% correct field completion levels. These thresholds give us a practical view of how much practitioner follow-up different returns still require. 我们可以通过了解 Tax AI 在无需后续修正的情况下完成申报表的准确度来量化这一改进。我们通过检查达到 75%、90% 或 100% 正确字段完成率的申报表占比来衡量准确度。上线时，只有四分之一的申报表达到 75% 的正确字段完成率，但在六周内，86% 的申报表都达到了这一标准。在 90% 和 100% 正确字段完成率水平上，系统表现出了更快的增长。这些阈值让我们能够实际了解不同申报表仍需要从业者多少后续跟进。

Early on, Tax AI handled simpler work, like W-2s and 1099s. As the season went on, it moved into more complex returns with K-1s, schedules, and harder edge cases. Each new capability saved more time per return than the last because the tasks it took on were harder and more time consuming to do manually. We continue to see ongoing progress today.早期，Tax AI 处理较简单的工作，如 W-2 和 1099 表格。随着报税季推进，它开始处理更复杂的申报表，包括 K-1、附表和更难的边缘情况。每一项新功能都比上一项节省更多时间，因为它承担的任务更难、手动处理更耗时。直到今天，我们仍持续看到进步。

Next, we’ll walk through how our teams co-engineered Tax AI to be self-improving by leaning on three critical pillars: 1) expert practitioner feedback, 2) production traces (a structured history from inputs through final output), and 3) a Codex-driven iteration loop based on tailored evals to enable continuous, faster product development. We hope our experience will be useful to other builders in domains where practitioner expertise is key to shaping the quality of the overarching system and the data running through it.接下来，我们将介绍我们的团队如何共同设计 Tax AI 使其具备自我改进能力，依托三大关键支柱：1）专家从业者的反馈，2）生产痕迹（从输入到最终输出的结构化历史记录），以及 3）基于定制评估的 Codex 驱动迭代循环，以实现持续、更快的产品开发。我们希望我们的经验对其他领域的构建者有所帮助，在那些领域中，从业者的专业知识是塑造整体系统质量及其运行数据质量的关键。

As we pushed into harder parts of tax preparation (K-1s, rental real estate schedules, and tax forms where values had to be reconciled across multiple source files), it became obvious that the real challenge was whether the product could make complex production failures visible, understandable, and actionable.随着我们深入到税务准备中更困难的部分（K-1、房地产租赁附表，以及需要在多个源文件之间核对数值的税表），真正的挑战变得显而易见：产品能否让复杂的生产故障变得可见、可理解且可处理。

In the early days of the product, most of the correction was manual. Practitioners could correct system errors, but the product did not capture the full context: a changed value before filing might reflect a true extraction miss, a mapping problem, missing product support, or expected workflow noise. Sorting those cases out still required follow-up from the engineering team. Engineers could use coding agents, but the system was not yet designed to use AI meaningfully inside an improvement loop. We did not have the signal to identify the right hill to climb.在产品早期，大部分修正都是手动的。从业者可以纠正系统错误，但产品未能捕捉完整上下文：提交前数值的变化可能反映了真正的提取遗漏、映射问题、产品支持缺失，或是预期的工作流程噪音。理清这些情况仍然需要工程团队的后续跟进。工程师可以使用编程智能体，但系统尚未被设计成在改进循环中有意义地使用 AI。我们没有信号来确定应该攻克哪个方向。

That led us to design the system around three pillars:这促使我们围绕三大支柱来设计系统：

1. Stay close to practitioners: The people doing the work need to steer what the product learns. Their intuition and understanding reveal which errors matter and help inform which parts of the workflow are worth focusing on next.贴近从业者：实际从事这项工作的人需要引导产品学习什么。他们的直觉和理解揭示了哪些错误重要，并帮助确定工作流程中哪些部分值得下一步重点关注。
2. Build the product so production creates evidence: The product has to capture more than just inputs and outputs; it needs to capture the full path from source material, to extracted fields and provenance, to downstream submission and expert correction.构建产品，使生产过程产生证据：产品必须捕捉的不仅仅是输入和输出；它需要捕捉从源材料，到提取的字段和来源信息，再到下游提交和专家修正的完整路径。
3. Create a Codex-driven improvement loop: Once production issues are visible and structured, they can become findings, tailored evals, and scoped engineering tasks. Codex can then help investigate, propose changes, validate them against targeted and regression evals, and move the product forward faster than a purely manual iteration cycle. 创建 Codex 驱动的改进循环：一旦生产问题变得可见且结构化，它们就可以成为研究发现、定制评估和范围明确的工程任务。然后 Codex 可以帮助调查、提出更改、针对目标评估和回归评估进行验证，并以比纯手动迭代周期更快的速度推进产品。

The rental properties example below shows how that loop works in practice, walking you through how a practitioner correction becomes a structured finding, then an eval target, and finally a Codex-scoped engineering task.下面的租赁物业示例展示了该循环在实践中如何运作，带你了解从业者的修正如何变成结构化的发现，然后成为评估目标，最终成为 Codex 范围内的工程任务。

Rental property income is reported on Schedule E of an individual tax return. From an engineering perspective, the task of extracting it is simple to describe but hard to do well. The system has to read messy source material (handwritten notes, emails, spreadsheets, and other client files), extract the rental-property fields the system can confidently map to the tax engine, and preserve enough evidence that a practitioner can approve or correct the result. The simplified example below shows what those source files and extracted outputs might look like.租赁物业收入在个人所得税申报表的附表 E 上报告。从工程角度来看，提取这项任务说起来简单，但做好很难。系统必须读取杂乱的源材料（手写笔记、电子邮件、电子表格和其他客户文件），提取系统可以自信映射到税务引擎的租赁物业字段，并保留足够的证据以便从业者可以批准或纠正结果。下面的简化示例展示了这些源文件和提取输出可能的样子。

A difference between the agent-predicted value and the actual value from the filed tax return might reflect a true extraction miss, but it could also be a practitioner preference, a value carried forward from a prior-year return in the tax engine, or a value introduced or changed elsewhere in the filing workflow. Practitioners helped us discern those cases so we could identify which actions required a practitioner correction or blocked a submission.智能体预测值与已提交纳税申报表实际值之间的差异可能反映了真正的提取遗漏，但也可能是从业者的偏好、税务引擎中往年申报表延续下来的数值，或是在申报工作流程其他环节引入或更改的数值。从业者帮助我们辨别这些情况，以便我们识别哪些操作需要从业者修正或阻碍了提交。

Because we could see these corrections in detail, we transformed the review process from a terminal, post-failure step into a continuous learning cycle. We designed the workflow to capture expert actions as structured data. Now, every intervention feeds the product's improvement loop by recording exactly what Tax AI proposed, what the practitioner modified, and what ultimately went into the filed return.因为我们可以详细看到这些修正，我们将审核流程从一个终结性的、失败后的步骤转变为一个持续的学习循环。我们将工作流程设计为以结构化数据捕捉专家操作。现在，每一次干预都通过准确记录 Tax AI 的提议、从业者修改的内容以及最终进入已提交申报表的内容，为产品的改进循环提供养分。

For a complex workflow like rental properties, the system has to preserve what happens between the source files and the filed return. Along that path, documents are organized, split, and classified; rental-property fields are extracted with citations back to the source material; those values are mapped into the tax engine; and practitioners may still correct them before filing. Those product-level traces make it possible to investigate where a failure occurred. To turn practitioner corrections into useful evaluation targets, the system processes them in three steps:对于租赁物业这样的复杂工作流程，系统必须保留从源文件到已提交申报表之间发生的所有事情。在这一路径上，文档被组织、拆分和分类；租赁物业字段被提取并附带引用指向源材料；这些数值被映射到税务引擎中；从业者仍可能在提交前进行修正。这些产品级别的痕迹使调查故障发生位置成为可能。为了将从业者修正转化为有用的评估目标，系统通过三个步骤处理它们：

• Capture the difference: Tax AI’s output is compared with the filed return to produce field-level review rows that capture the expected value, predicted value, and whether the difference appears actionable.捕捉差异：将 Tax AI 的输出与已提交的申报表进行比较，生成字段级别的审核行，记录预期值、预测值以及该差异是否看起来可处理。
• Group related failures: Similar review rows are grouped to separate recurring product failures from expected workflow noise. For example, repeated practitioner corrections might show that Tax AI often misses fair-rental-day fields, mishandles “other expenses,” or confuses multiple rental properties across the same source package.分组相关故障：将相似的审核行分组，以区分反复出现的产品故障和预期的工作流程噪音。例如，反复的从业者修正可能表明 Tax AI 经常遗漏公平租赁天数字段、错误处理“其他费用”，或混淆同一源文件包中的多处租赁物业。
• Turn repeated patterns into eval targets: Once reviewed and measured, repeated findings become clear eval targets for Codex to improve.将重复出现的模式转化为评估目标：经过审核和衡量后，反复出现的发现会成为明确的评估目标，供 Codex 改进。

The third pillar is creating an engineering loop capable of acting on these new evals. This is where Codex becomes central.第三大支柱是创建一个能够对这些新评估采取行动的工程循环。这就是 Codex 成为核心之处。

Suppose our eval pipeline flags that Tax AI consistently misses the "fair rental days" field, while practitioners reliably fill it in. Because this finding has already been packaged into a targeted eval set, with representative source packages and expected outputs, Codex can investigate the root cause directly within the product scaffold.假设我们的评估流程标记出 Tax AI 始终遗漏“fair rental days”字段，而从业者总能可靠地填写它。由于这一发现已被打包进一个有针对性的评估集，其中包含具有代表性的源文件包和预期输出，Codex 可以直接在产品框架内调查根本原因。

Codex isn’t working solely with a sub-par final output. It inspects the trace, eval, repo, and skills together:Codex 不仅仅是在处理一个欠佳的最终输出。它会同时检查痕迹、评估、代码库和技能：

• Investigate the pipeline: Inspect source packages, extraction schemas, mapper behavior, and code paths to determine whether the issue is an unsupported field, a missed extraction pattern, a source-selection problem, a mapper gap, or a grader issue.调查流程：检查源文件包、提取模式、映射器行为和代码路径，以确定问题是不支持的字段、遗漏的提取模式、源选择问题、映射器缺陷，还是评分器问题。
• Implement targeted fixes: Extend the extraction schema, improve source selection for rental-property documents, update the tax-engine mapper, or refine the grader if expected workflow noise is being counted as a failure.实施有针对性的修复：扩展现有提取模式、改进租赁物业文档的源选择、更新税务引擎映射器，或者如果预期的工作流程噪音被算作故障，则优化评分器。
• Validate and propose: Rerun the targeted eval, run broader regression suites, and surface a candidate pull request for engineering review.验证并提出：重新运行有针对性的评估，运行更广泛的回归测试套件，并提交候选拉取请求供工程审核。
• Close the loop: Turn a recurring practitioner correction into a measurable engineering task. If the evidence is ambiguous or not safely automatable, the case routes back to the product team instead of being forced through the loop.闭环：将反复出现的从业者修正转化为可衡量的工程任务。如果证据不明确或无法安全地自动化，该案例将退回给产品团队，而不是被强制推入循环。

The rental property example is emblematic of a broader reusable pattern: using production artifacts and traces to improve an agent’s capabilities. Given reviewed findings from production data, source traces, expected tax-engine output, relevant code examples, and eval commands as a set of inputs, Codex can materially improve on performance and accuracy over weeks and months. This builds on the principles described in our work on harness engineering and Symphony, which walk-through how to make tasks legible to Codex, provide scoped context and tools, and keep validation and human review part of the environment. 租赁物业示例象征着一个更广泛的可复用模式：利用生产产物和痕迹来提升智能体的能力。如果以生产数据中的已审核发现、源痕迹、预期的税务引擎输出、相关代码示例和评估命令作为一组输入，Codex 可以在数周和数月内实质性提升性能和准确度。这建立在我们关于 harness engineering 和 Symphony 的研究所描述的原则之上，这些研究介绍了如何让任务对 Codex 清晰可读、提供有范围的上下文和工具，并将验证和人工审核保留为环境的一部分。

That evidence does not become a Codex task automatically. A practitioner correction may reflect an extraction miss, a mapping issue, unsupported product behavior, tax judgment, or expected workflow noise. Only after repeated differences have been reviewed and grouped into an actionable finding does the system turn them into a bounded task with a clear success condition.这些证据不会自动成为 Codex 任务。从业者的修正可能反映了提取遗漏、映射问题、不支持的产品行为、税务判断，或是预期的工作流程噪音。只有在反复出现的差异被审核并归类为可操作的发现后，系统才会将它们转化为具有明确成功条件的有边界任务。

We apply this automation to a bounded layer of the product. This layer performs extraction and maps source documents into tax workflows. Engineers remain responsible for architecture, product decisions, and shipping. Practitioners steer the improvement loop through the work they already do: correcting extracted values, reviewing returns, and approving final filings.我们将这种自动化应用于产品的一个有边界层面。该层面执行提取并将源文档映射到税务工作流中。工程师仍然负责架构、产品决策和发布。从业者通过他们已经在做的工作来引导改进循环：纠正提取的数值、审核申报表以及批准最终提交文件。

For Codex, the result is not a vague alert but a scoped engineering task with evidence, editable product surfaces, and explicit validation gates. The context for a representative rental property task can be summarized as follows:对于 Codex 来说，结果不是模糊警报，而是一个有范围的工程任务，附带证据、可编辑的产品层和明确的验证门槛。一个典型的租赁物业任务的上下文可以概括如下：

The same loop applies beyond rental properties. Rental properties took about six weeks and substantial engineering oversight to reach 90% precision and recall, but that work produced reusable abstractions, review artifacts, eval conventions, and implementation patterns that made it easier to support similarly complex schedules such as Schedule C and Schedule A.同样的循环也适用于租赁物业之外。租赁物业花了大约六周时间和大量的工程监督才达到 90% 的精确率和召回率，但这项工作产生了可复用的抽象、审核产物、评估惯例和实现模式，使得支持类似复杂的附表（如附表 C 和附表 A）变得更加容易。

Tax AI proves a path to building self-improving agents. Practitioners generate high-value feedback signals by delivering the service. Product workflows preserve those signals as structured evidence. Eval-backed engineering systems validate improvements before they reach production, and an agent-powered loop keeps the system in a continuous self-improving flow. Tax AI 证明了构建自我改进智能体的路径。从业者通过提供服务产生高价值的反馈信号。产品工作流程将这些信号保留为结构化证据。由评估支持的工程系统在改进进入生产环境之前对其进行验证，而由智能体驱动的循环使系统保持在持续自我改进的流程中。

Thrive Holdings’ structure allows us to replicate this environment in specific industries. Holdings is both an owner and operator, so our combined engineering teams are able to work directly with practitioners and production data from inside businesses like Crete, not as a vendor but as partners. This means the technology, the product, and the service all sit under one roof to help us move faster and build exceptional products.Thrive Holdings 的结构使我们能够在特定行业复制这一环境。Holdings 既是所有者也是经营者，因此我们联合的工程团队能够直接与从业者合作，并在 Crete 这样的企业内部获取生产数据，我们不是作为供应商，而是作为合作伙伴。这意味着技术、产品和服务都在同一个屋檐下，帮助我们更快地行动并打造出色的产品。

One senior accountant who spent 180 hours on tax prep last year spent only 15 hours on it this year. She put that time in part toward calling every one of her clients and walking them through their returns, a level of high touch service that wasn’t possible a year ago. The rest of that time she used to take on new clients and expand to new service offerings.一位去年花了 180 小时做税务准备工作的高级会计师，今年只花了 15 小时。她把这部分时间用来给每一位客户打电话，并逐一向他们讲解申报表，这种高接触度的服务在一年前是不可能实现的。其余时间她用来承接新客户并拓展新的服务。

Together, our teams are now using the same three-part design from Tax AI as a blueprint for building workflows in other domains across Thrive Holdings⁠(opens in a new window); accounting workflows such as bookkeeping and audit, and operational workflows such as IT help desk automation. Across domains and industries, the broader promise of self-improving agents holds. The best agents are steered by people to learn to become more capable, more trusted, and more valuable over time.目前，我们的团队正在共同使用 Tax AI 中的同款三部分设计作为蓝图，在 Thrive Holdings（在新窗口中打开）的其他领域构建工作流；例如记账和审计等会计工作流，以及 IT 服务台自动化等运营工作流。跨越不同领域和行业，自我改进智能体的更广泛前景依然成立。最出色的智能体由人引导，学习变得更有能力、更值得信赖，并随着时间推移变得更有价值。

To learn more about the OpenAI team that worked on this project, get in touch.如需了解参与该项目的 OpenAI 团队更多信息，请与我们联系。