- We’ve built an AI agent that acts as a secondary expert for a given domain, making deep specialist knowledge readily available and preserved for anyone in an organization to access, share, and build upon.我们构建了一个 AI 智能体,它能充当特定领域的辅助专家,使深度的专业知识变得触手可及,并得以保存,供组织内的任何人访问、共享和在此基础上进行开发。
- This is not a typical domain-specific agent. Its novelty comes from integrating two layers:
- A structured, auditable knowledge architecture separates what the agent knows from how it reasons. 一种结构化、可审计的知识架构将智能体的“所知”与其“推理方式”分离开来。
- A self-improvement loop then compiles expert feedback into verified, regression-tested updates without model retraining. 随后,一个自我改进循环将专家反馈汇编成经过验证、回归测试的更新,而无需对模型进行重新训练。
- Together, these two layers turn one-off expert corrections into permanent, compounding institutional memory, and the pattern is designed to generalize to other domains governed by retrievable text rather than model weights.这两层架构共同将一次性的专家修正转化为永久的、不断积累的组织记忆;该模式旨在推广到其他由可检索文本而非模型权重所管理的领域。
- This system is saving domain subject matter experts (SME)s at Meta substantial time, allowing them to focus more on the work where their knowledge matters most.该系统正在为 Meta 的领域主题专家(SME)节省大量时间,使他们能够更专注于那些最能体现其知识价值的工作。
Many large organizations have the same problem when it comes to specialist knowledge. While some of it is written down in the form of models, playbooks, checklists, and frameworks, the most valuable specialist knowledge lives in people’s heads and rarely gets captured anywhere durable. In compliance domains, for example, the same types of questions can arise across hundreds of product reviews, expert assessments take days of manual research, and inconsistency between assessments creates real organizational risk.许多大型组织在处理专业知识时都面临同样的问题。虽然部分知识以模型、手册、核对清单和框架的形式被记录下来,但最有价值的专业知识往往存在于人们的头脑中,很少被持久地记录。例如,在合规领域,同样的疑问可能出现在数百次产品审查中,专家评估需要数天的手动研究,而评估结果之间的不一致会造成真正的组织风险。
It’s not uncommon for experts to spend more time answering routine questions than on genuinely novel and ambiguous work where their judgment matters most. We need systems that can capture how an organization’s experts reason and make that knowledge available to everyone who needs it, so that expertise is easier to share, build on, and preserve.
We set about solving this challenge by codifying institutional intelligence into an AI agent for a specific compliance domain. The agent combines a knowledge system that acts as the organization’s “second brain,” a reasoning layer that mirrors how domain experts actually think, and an automated improvement pipeline that compounds expert effort permanently. The patterns generalize to any enterprise domain with deep specialist knowledge, whether that is finance, security, or engineering. 专家花费在回答常规问题上的时间多于真正新颖且模糊、需要其判断力的工作,这种情况并不罕见。我们需要能够捕捉组织专家推理方式的系统,并使这些知识可供所有需要的人使用,从而让专业知识更易于共享、构建和保存。
我们通过将组织情报编码到一个特定合规领域的 AI 智能体中来解决这一挑战。该智能体结合了一个充当组织“第二大脑”的知识系统、一个反映领域专家实际思考方式的推理层,以及一个永久积累专家努力的自动化改进流程。这些模式适用于任何拥有深厚专业知识的企业领域,无论是金融、安全还是工程。
The Architecture at a Glance架构概览
Off-the-shelf LLMs provide a strong foundation, but they often need deeper institutional context to be fully effective in specialist domains. Without that grounding, a general purpose model has limited value given it will not be able to distinguish between what an organization could do (a summary of general information) and what it should consider doing (based on historic positions, company direction, business context, etc.). In high-stakes domains, closing this gap requires supplying the model with the organization’s own knowledge and priorities so its analysis reflects how the organization actually reasons.现成的 LLM(大语言模型)提供了强大的基础,但它们通常需要更深厚的组织背景才能在专业领域中充分发挥作用。如果没有这种基础,通用模型价值有限,因为它无法区分组织“可以做什么”(一般信息的总结)和“应该考虑做什么”(基于历史立场、公司方向、业务背景等)。在高风险领域,弥合这一差距需要为模型提供组织自身的知识和优先级,以便其分析能够反映组织实际的推理方式。
The system we’ve designed has four layers, each solving a distinct problem:我们设计的系统包含四个层级,每一层解决一个具体问题:

These layers depend on each other. The knowledge system’s file structure makes automated editing possible. The reasoning layer’s explicit procedures make failure attribution tractable. The evaluation framework gates every change. And the improvement loop feeds back into both knowledge and reasoning. Remove any one layer and the others degrade.这些层级相互依赖。知识系统的文件结构使得自动化编辑成为可能;推理层的显式过程使得故障归因变得可追溯;评估框架为每一次变更把关;改进循环则反馈给知识层和推理层。移除其中任何一层,其他层级都会退化。
Building the Organizational Second Brain构建组织级的“第二大脑”
Large organizations can accumulate thousands of documents as a byproduct of expert work. It is tempting to treat those documents as organizational knowledge, but the real knowledge is implicit: how experts reason, what they prioritize, and how they resolve ambiguity. An agent that retrieves document chunks at inference time has to re-derive that reasoning from raw sources on every run, which is slow, error-prone, and inconsistent.大型组织在专家工作过程中会积累数千份文档。人们很容易将这些文档视为组织知识,但真正的知识是隐性的:专家如何推理、他们优先考虑什么,以及他们如何解决模糊性。一个在推理时检索文档片段的智能体必须在每次运行时从原始来源重新推导推理过程,这既缓慢、容易出错,也不一致。
We make that implicit knowledge explicit ahead of time. A long-running offline process reasons through source documents and distills them into structured knowledge files – curated statements of how the organization interprets its domain, with constraints, boundaries, and routing implications made machine-readable. 我们提前将这些隐性知识显性化。一个长期运行的离线流程会对源文档进行推理,并将它们提炼成结构化的知识文件——即经过整理的陈述,说明组织如何解读其领域,并将约束、边界和路由含义转化为机器可读的形式。
Most significantly, that knowledge then forms the basis of a feedback loop that allows the agent to learn from and implement feedback from human experts without the underlying model having to be retrained.最重要的是,这些知识随后构成了一个反馈循环的基础,允许智能体学习并实施来自人类专家的反馈,而无需对底层模型进行重新训练。
The industry has converged on a similar idea. Andrej Karpathy’s LLM Wiki structures agent knowledge as a navigable graph of files, and Google’s Open Knowledge Format standardizes this for cross-agent interoperability. The shared insight is that knowledge should be pre-extracted, explicitly structured, and progressively disclosed rather than re-derived on every query. We extended these principles into a system where citation fidelity and institutional consistency are non-negotiable, organizing 200+ files into a strict taxonomy:业界已趋向于一个类似的理念。Andrej Karpathy 的 LLM Wiki 将智能体知识构建为可导航的文件图谱,而 Google 的 Open Knowledge Format 则将其标准化以实现跨智能体的互操作性。共同的见解是,知识应当被预先提取、显式结构化,并逐步披露,而不是在每次查询时重新推导。我们将这些原则扩展到一个系统,其中引用保真度和组织一致性是不可妥协的,并将 200 多个文件组织成严格的分类体系:
- Position files capture authoritative organizational stances: how the organization has decided to interpret a given domain question, along with its constraints, boundary conditions, and machine-actionable routing implications that tell the reasoning layer when to apply it.立场文件(Position files)记录权威的组织立场:组织决定如何解读特定的领域问题,以及其约束、边界条件和机器可操作的路由含义,这些含义告知推理层何时应用该立场。
- Taxonomy and vocabulary files act as an authoritative glossary for the terms the organization uses to describe its domain, such as entity types, activity categories, and classification tiers. Each is maintained as a single source of truth so the agent and the organization use language consistently.分类和词汇文件(Taxonomy and vocabulary files)充当组织用于描述其领域的术语的权威词汇表,例如实体类型、活动类别和分类级别。每一项都作为单一事实来源进行维护,以确保智能体和组织使用语言的一致性。
- Routing indexes map input characteristics to the relevant positions and procedures, determining which files apply without relying on embedding similarity alone. This makes retrieval deterministic and auditable.路由索引(Routing indexes)将输入特征映射到相关的立场和程序,从而在不依赖嵌入相似度的情况下确定适用哪些文件。这使得检索过程具有确定性和可审计性。
- Gateway files define threshold tests the agent must pass before entering an analytical domain, preventing it from applying specialized knowledge where it does not belong.网关文件(Gateway files)定义了智能体在进入分析领域前必须通过的阈值测试,防止其在不适用的情况下应用专业知识。
Every file declares its dependencies (depends_on) and consumers (referenced_by) in YAML frontmatter, forming a bidirectional dependency graph. When one file changes, you can trace exactly what else might be affected, which matters when the self-improvement loop proposes automated edits.每个文件都在 YAML 前置元数据(frontmatter)中声明其依赖项(depends_on)和消费者(referenced_by),形成一个双向依赖图。当一个文件发生变化时,你可以精确追踪到其他可能受影响的内容,这在自我改进循环提出自动化编辑建议时至关重要。

Organizing Knowledge by Density and Usage Frequency按密度和使用频率组织知识
A key architectural decision is how to partition knowledge between the curated wiki and supplementary retrieval (RAG). We split on information density and expected usage frequency.一个关键的架构决策是如何在精选 Wiki 和补充检索(RAG)之间划分知识。我们根据信息密度和预期使用频率进行划分。
High-density, frequently referenced sources go into the wiki: Distilled files capturing how the organization reasons, such as positions, decision frameworks, boundary examples, and strategic interpretations. The agent consults these on nearly every turn. Because they encode the organization’s evolving thinking, they need to stay current, and the wiki structure makes them easy to update, version, and validate.高密度、经常引用的来源进入 Wiki:提炼出的文件记录了组织的推理方式,例如立场、决策框架、边界示例和战略解读。智能体几乎在每次交互时都会查阅这些内容。由于它们编码了组织不断演进的思维,因此需要保持最新,而 Wiki 结构使它们易于更新、版本控制和验证。
Sparse, situationally relevant sources are served through semantic or lexical search (RAG): documents that matter deeply when they apply but are not needed in detail on most runs, such as detailed reference material, individual product specifications, historical decision records, and niche external knowledge. Loading all of them into the wiki would bloat the system and dilute attention.稀疏的、情境相关的来源通过语义或词法搜索(RAG)提供:这些文档在适用时非常重要,但在大多数运行中不需要详细信息,例如详细参考资料、个人产品规格、历史决策记录和利基外部知识。将所有这些内容加载到 Wiki 中会使系统臃肿并分散注意力。
The result is that the agent’s core reasoning is always grounded in the most refined, current organizational knowledge, while it can still reach for supporting evidence when a scenario demands it. The combination produces an organizational second brain that encodes how the organization interprets and applies information, rather than only where to find it.结果是,智能体的核心推理始终扎根于最精炼、最新的组织知识中,同时在场景需要时仍能获取支持证据。这种结合产生了一个组织级的“第二大脑”,它编码了组织如何解读和应用信息,而不仅仅是去哪里寻找信息。
Expert Reasoning via Composable Recipes通过可组合的“配方”进行专家推理
Knowledge alone is not enough. Domain experts do not simply recall facts, they follow structured methodologies: a financial analyst works through a valuation model step by step, a security engineer follows a threat modeling procedure. The challenge is capturing those methodologies in a form an LLM can execute reliably.仅有知识是不够的。领域专家不仅仅是回忆事实,他们遵循结构化的方法论:金融分析师一步步完成估值模型,安全工程师遵循威胁建模程序。挑战在于以 LLM 可以可靠执行的形式捕捉这些方法论。
We solve this with composable procedures we call recipes. Where knowledge files are declarative, recipes are imperative. Each one prescribes a multi-step analytical workflow, specifying what to examine first, which knowledge to load at each step, what decision procedures to follow, and what constitutes a complete analysis.我们通过称为“配方”(recipes)的可组合程序来解决这个问题。知识文件是声明性的,而配方是命令性的。每个配方规定了一个多步骤的分析工作流,指定首先检查什么、在每一步加载哪些知识、遵循什么决策程序,以及什么构成完整的分析。
The critical design choice is separating what the agent knows from how it reasons. Recipes reference knowledge files but contain no domain facts; knowledge files state positions but prescribe no procedures. This means:关键的设计选择是将智能体的“所知”与其“推理方式”分离开来。配方引用知识文件但不包含领域事实;知识文件陈述立场但不规定程序。这意味着:
- Adding an organizational position means adding a knowledge file and updating a routing index. No recipe changes.添加组织立场意味着添加一个知识文件并更新路由索引。无需更改配方。
- Fixing a flaw in the agent’s methodology means editing a recipe. No knowledge files change.修复智能体方法论中的缺陷意味着编辑配方。无需更改知识文件。
- Failures attribute cleanly to one layer. Was the knowledge wrong, or the procedure?故障可以清晰地归因于某一层。是知识错了,还是程序错了?
Recipes compose into pipelines, much like a head chef’s master recipe for a dinner service delegates to sub-recipes for each component (the sauce, the protein, the garnish) without containing those details itself. Our top-level routing recipe examines the input and selects which downstream recipes to invoke, each handling one analytical phase.配方可以组合成管道,就像主厨为晚餐服务制定的总配方,会委托给每个组件(酱汁、蛋白质、装饰)的子配方,而自身不包含这些细节。我们的顶级路由配方检查输入并选择调用哪些下游配方,每个配方处理一个分析阶段。
This is also what enables progressive disclosure. Rather than front-loading a monolithic instruction set covering every possible scenario, each recipe step carries only the instructions and knowledge relevant to that phase. Early versions used a single flat instruction file and loaded all sources via semantic search, pulling a large volume of mixed-relevance files into the context window on every run. After restructuring into recipe-driven stages, each query touches only a small, targeted subset, cutting tokens consumed per turn by around 80%. Context windows are finite and attention degrades with volume, so delivering the right instructions at the right time directly improves reasoning quality.这也是实现“逐步披露”的原因。与其预先加载一个涵盖所有可能场景的单一指令集,每个配方步骤仅携带该阶段相关的指令和知识。早期版本使用单个平面指令文件,并通过语义搜索加载所有来源,导致每次运行时都会将大量相关性不一的文件拉入上下文窗口。在重构为配方驱动的阶段后,每次查询仅触及一小部分目标子集,使每次交互消耗的 token 减少了约 80%。上下文窗口是有限的,注意力会随容量增加而退化,因此在正确的时间提供正确的指令直接提高了推理质量。
Keeping Humans in Control保持人类的控制
Human experts stay in control of this system throughout. The agent accelerates and structures their work; it does not replace their judgment or their authority over the outcome. 人类专家始终保持对该系统的控制。智能体加速并结构化了他们的工作;它不会取代他们的判断或对结果的权威性。
We enforce this through two mechanisms: 我们通过两种机制强制执行这一点:
Checkpoints are defined points in the analysis where the agent surfaces its intermediate reasoning for expert review before proceeding, and the expert can confirm, correct, or redirect. 检查点(Checkpoints)是分析中定义的点,智能体在继续之前会呈现其中间推理供专家审查,专家可以确认、纠正或重定向。
Escalations trigger when the agent hits genuine ambiguity, whether from underspecified inputs or evidence that supports more than one defensible reading. Rather than forcing a resolution, it hands the question to the expert, whose choice determines the path the analysis takes.升级机制(Escalations)在智能体遇到真正的模糊性时触发,无论是由于输入定义不明确,还是因为证据支持多种可辩护的解读。它不会强行解决问题,而是将问题提交给专家,由专家的选择决定分析的路径。
Checkpoints and escalations serve three purposes simultaneously:检查点和升级机制同时服务于三个目的:
- Quality and direction control: Experts catch errors before they compound downstream, and keep the analysis on the path they consider most relevant.质量和方向控制:专家在错误蔓延到下游之前发现它们,并使分析保持在他们认为最相关的轨道上。
- Training signal: Every correction and every escalation becomes input for the self-improvement loop.训练信号:每一次纠正和每一次升级都成为自我改进循环的输入。
- Trust calibration: Experts build confidence incrementally by observing the agent’s reasoning rather than just its final output, and by seeing it flag uncertainty instead of masking it.信任校准:专家通过观察智能体的推理过程(而非仅仅是最终输出),以及看到它标记不确定性而非掩盖它,从而逐步建立信任。
The more consequential the decision, the more this matters, which is why we recommend keeping a human in the loop by default across domains like compliance, financial risk assessment, security review, and engineering safety.决策后果越严重,这一点就越重要,这就是为什么我们建议在合规、金融风险评估、安全审查和工程安全等领域默认保持人类参与。
The Self-Improvement Flywheel自我改进的飞轮
We consider the self-improvement flywheel to be the most distinctive part of this system. While the structured knowledge system and composable recipes produce a system that is legible by humans and agents, testable, and modular, the number of interdependent files make manual maintenance impossible to scale. When domain experts provide feedback to an agent that feedback has to be translated into precise file edits. This process can take weeks because it requires understanding the full dependency graph, verifying nothing else breaks, and validating the fix actually works.我们认为自我改进飞轮是该系统最独特的组成部分。虽然结构化的知识系统和可组合的配方产生了一个人类和智能体可读、可测试且模块化的系统,但相互依赖的文件数量使得人工维护无法扩展。当领域专家向智能体提供反馈时,该反馈必须转化为精确的文件编辑。这个过程可能需要数周时间,因为它需要理解完整的依赖图,验证没有其他内容被破坏,并验证修复方案确实有效。
There has been a large body of work – from RAG memory systems to model-weight knowledge editing – devoted to addressing how agents store, retrieve, and update knowledge. But much less attention has gone toward keeping a document-based institutional knowledge base correct as it grows and as expert positions evolve. Agents that auto-draft their own fixes are increasingly common; but we haven’t seen this level of validation rigor applied to a structured knowledge base without model retraining. 业界已经开展了大量工作——从 RAG 记忆系统到模型权重知识编辑——致力于解决智能体如何存储、检索和更新知识。但对于如何随着文档型机构知识库的增长以及专家立场的演变来保持其正确性,关注度则少得多。自动起草修复方案的智能体正变得越来越普遍;但我们尚未看到这种级别的验证严谨性在不进行模型重训练的情况下应用于结构化知识库。
We treat that maintenance as a compilation problem and automate it. Every expert correction moves through four phases:
我们将这种维护视为一个编译问题并将其自动化。每一次专家纠正都会经历四个阶段:
- Diagnose expert feedback into actionable issues with their root cause.诊断:将专家反馈转化为可操作的问题及其根本原因。
- Compile issues into minimal verified edits.编译:将问题编译成最小化的已验证编辑。
- Validate that fixes work without regressions.验证:确认修复方案有效且无回归。
- Have domain experts review them.审查:由领域专家进行最终审核。
Once the loop completes, the regression test suite is enriched with the issue that was just fixed so that future updates preserve this behavior.一旦循环完成,回归测试套件就会加入刚刚修复的问题,以便未来的更新能保留这一行为。

Diagnosis: Attributing Each Correction to a Root Cause诊断:将每一次纠正归因于根本原因
Raw expert feedback comes from conversation traces where domain SMEs interacted with the agent and provided corrections. The diagnosis phase extracts structured signals from these conversations.原始专家反馈来自领域 SME 与智能体交互并提供纠正的对话记录。诊断阶段从这些对话中提取结构化信号。
Our first approach classified feedback by conversational form. If the expert provided information, it must be a knowledge gap; if they redirected the agent, it must be a procedure problem. This heuristic failed because conversational form is a poor proxy for root cause. An expert correcting a conclusion might be exposing a knowledge gap, a recipe flaw, or a genuine ambiguity.我们的第一种方法是按对话形式对反馈进行分类。如果专家提供了信息,那一定是知识缺口;如果他们重定向了智能体,那一定是程序问题。这种启发式方法失败了,因为对话形式是根本原因的糟糕代理。专家纠正结论可能暴露了知识缺口、配方缺陷或真正的模糊性。
The working approach separates extraction from classification. First, extract every substantive signal from the expert alongside the agent’s full knowledge manifest (every file loaded, when, and how used). Second, read the actual knowledge files and apply a single attribution test: Could the agent have reached the correct conclusion from its source materials?
当前的工作方法将提取与分类分开。首先,从专家那里提取每一个实质性信号,并结合智能体的完整知识清单(加载的每个文件、时间和使用方式)。其次,阅读实际的知识文件并应用单一归因测试:智能体是否能从其源材料中得出正确的结论?
- If the materials contained the right answer but the agent still erred: recipe problem.如果材料包含正确答案但智能体仍然出错:配方问题。
- If the materials did not contain the right answer: knowledge gap.如果材料不包含正确答案:知识缺口。
- If experts themselves disagree on the right answer: ambiguity, flagged for human discussion.如果专家本人对正确答案存在分歧:模糊性,标记为供人类讨论。
Compilation: Surgical Multi-Agent Edits编译:外科手术式的多智能体编辑
The compiler translates each diagnosed issue into minimal file edits. Sub-agents analyze impact in parallel, examining cross-references, conflicts with existing positions, token budget impact, test coverage, and duplication risk.
编译器将每个诊断出的问题翻译成最小化的文件编辑。子智能体并行分析影响,检查交叉引用、与现有立场的冲突、token 预算影响、测试覆盖率和重复风险。
Two design choices make this trustworthy:
两个设计选择使其值得信赖:
Independent adversarial review. A separate agent, running in a fresh context with no knowledge of the improvement rationale, receives only the proposed diffs to the knowledge base. Its job is to find problems such as contradictions introduced, edge cases broken, or positions undermined. Because it shares no context with the proposing agents, it cannot inherit their blind spots.独立的对抗性审查。一个单独的智能体在全新的上下文中运行,不了解改进的基本原理,仅接收对知识库的拟议差异(diffs)。它的任务是发现问题,例如引入的矛盾、破坏的边缘情况或被削弱的立场。因为它与提议智能体不共享上下文,所以不会继承它们的盲点。
Deterministic structural validation. A linter catches issues programmatically, dangling cross-references, file size budget violations, identifier collisions, and dependency cycles. This layer is not probabilistic. It passes or fails.确定性的结构验证。Linter(代码检查器)以编程方式捕获问题,如悬空的交叉引用、文件大小预算违规、标识符冲突和依赖循环。这一层不是概率性的。它要么通过,要么失败。
Evaluation: Proving the Fix Works评估:证明修复方案有效
Every proposed change goes through a two-stage validation:
每一个拟议的变更都要经过两阶段验证:
Targeted replay runs the agent on the original scenario that triggered the feedback. The agent does not know it is being tested. A separate judge evaluates the new output against the original expert feedback without knowing what was changed. This deliberately blind design prevents confirmation bias. If targeted replay fails, compilation is retried.针对性重放(Targeted replay):在触发反馈的原始场景上运行智能体。智能体不知道自己正在被测试。一个独立的裁判在不知道变更内容的情况下,根据原始专家反馈评估新输出。这种刻意的盲测设计防止了确认偏误。如果针对性重放失败,则重新尝试编译。
Regression testing runs multiple benchmarks for that domain, which are usually structured test suites of Q&A pairs. For analytical domains whether there might be multiple correct answers, an independent LLM judge gives each test case a pass/fail based on certain criteria. The agent is run in parallel, independent sessions against the benchmark questions, and regressions in performance are detected. If regression testing fails, compilation is retried with an updated prompt describing where the agent regressed, along with the original issue and attempted fix.回归测试(Regression testing):运行该领域的多个基准测试,这些测试通常是结构化的问答对测试套件。对于可能存在多个正确答案的分析领域,独立的 LLM 裁判根据特定标准给每个测试用例打分(通过/失败)。智能体在并行、独立的会话中针对基准问题运行,并检测性能回归。如果回归测试失败,则使用更新后的提示词重新尝试编译,提示词描述了智能体回归的位置,以及原始问题和尝试的修复方案。
Landing and Enrichment: Compounding Returns落地与丰富:复合收益
The output of the pipeline is a pull request (diff) with a complete audit trail. A human expert reviews a proven fix rather than debugging a raw failure. Once approved and landed (the knowledge file or recipe is updated), the original failing scenario and its validated correct answer are automatically added to the regression test suite. This means every fix permanently raises the bar and future changes to the knowledge system must preserve the behavior that was just corrected.管道的输出是一个带有完整审计追踪的合并请求(diff)。人类专家审核的是一个已验证的修复方案,而不是调试原始故障。一旦获得批准并落地(知识文件或配方已更新),原始失败的场景及其验证过的正确答案会自动添加到回归测试套件中。这意味着每一次修复都永久性地提高了标准,未来对知识系统的任何变更都必须保留刚刚纠正的行为。
Results结果
After three development sprints spanning six weeks, the system achieved:经过为期六周的三个开发冲刺,系统实现了:
- Domain SMEs rated agent outputs useful almost all the time, a significant improvement from early versions where outputs frequently required substantial rework.领域 SME 认为智能体输出几乎在所有情况下都有用,这比早期版本有了显著改进,早期版本的输出通常需要大量返工。
- Days to minutes reduction in individual assessment time.个人评估时间从数天缩短至数分钟。
- Automated self-improvement producing validated knowledge edits at a rate that previously required full engineering sprints.自动化自我改进以以前需要完整工程冲刺的速度产生经过验证的知识编辑。
- Zero regressions across improvement cycles, with every fix automatically strengthening the regression suite.在改进周期中实现零回归,每一次修复都会自动加强回归测试套件。
- Domain experts consistently reported the agent handles the vast majority of the analytical work, allowing them to focus on the genuinely ambiguous cases that require human judgment.领域专家一致报告称,智能体处理了绝大多数分析工作,使他们能够专注于需要人类判断的真正模糊的案例。
Applying This Architecture应用此架构
The specific domain we built this for required synthesizing dozens of sources (both internal positions and external material) into risk-weighted assessments. But the architecture is domain-independent. It applies wherever:
我们为此构建的特定领域需要将数十个来源(内部立场和外部材料)综合成风险加权评估。但该架构是领域无关的。它适用于以下任何情况:
- Specialist knowledge lives as tribal knowledge in experts’ heads.专业知识以部落知识的形式存在于专家的头脑中。
- Consistency across assessments matters.评估之间的一致性很重要。
- The volume of work exceeds available expert capacity.工作量超过了现有的专家容量。
- Off-the-shelf LLMs produce inadequate analysis.现成的 LLM 产生的分析不足。
Concrete domains where this pattern fits include regulatory compliance, protocol adherence, financial risk assessment, security review, engineering standards compliance, and procurement evaluation. The common thread is that organizations need AI systems with genuine institutional expertise, not just general knowledge.适用此模式的具体领域包括监管合规、协议遵守、金融风险评估、安全审查、工程标准合规和采购评估。共同点是组织需要拥有真正机构专业知识的 AI 系统,而不仅仅是通用知识。
The requirements for adopting this architecture are:
采用此架构的要求是:
- A structured knowledge system with explicit file boundaries, cross-references, and a dependency graph (the organizational second brain for the domain).一个具有明确文件边界、交叉引用和依赖图的结构化知识系统(该领域的组织级“第二大脑”)。
- A procedural layer that separates domain knowledge from analytical methodology (recipes).一个将领域知识与分析方法论(配方)分离开来的程序层。
- An automated evaluation suite that grows with each improvement cycle.一个随每个改进周期而增长的自动化评估套件。
- Human-in-the-loop checkpoints calibrated to the domain’s risk tolerance.根据领域风险承受能力进行校准的人机协同检查点。
The deeper principle is straightforward: keep the complexity in text files that are readable by both humans and agents, rather than fine-tuned model weights. Every improvement is a text edit that a domain expert can review in 30 seconds. Every change is version-controlled, diffable, and reversible. The compilation pipeline is sophisticated, but its outputs are always transparent.
更深层的原则很简单:将复杂性保留在人类和智能体都可读的文本文件中,而不是微调的模型权重中。每一次改进都是领域专家可以在 30 秒内审核的文本编辑。每一次变更都是版本控制的、可差异化的且可逆的。编译管道虽然复杂,但其输出始终是透明的。
The goal is a system where expert effort compounds permanently. Every expert interaction makes the system better. Every correction persists as a verified improvement. The organization’s collective knowledge stops being trapped in individuals and starts being available, consistently and at scale, to everyone who needs it.目标是一个专家努力可以永久积累的系统。每一次专家交互都使系统变得更好。每一次纠正都作为已验证的改进而持久存在。组织的集体知识不再困在个人身上,而是开始一致且大规模地提供给所有需要的人。
Acknowledgements致谢
The authors would like to express our gratitude to the contributions of the following people, who have played a crucial role in developing this system. In particular, we would like to extend special thanks to (in last name alphabetical order): Cecilia Baek, Philipp Kaufold, Cat Hughes, Suzanne Leijten, Michael Marcusa, Jordi Mola, Timothy Neo, Elliott Prentiss, Laia Reyes, John Ross, Julio Santil, Taylor Wilson Thomas, Mansi Tripathi, Nikhil Shanbhag, Madeleine Vos, and Jackie Zajac.作者谨向在开发该系统过程中发挥关键作用的以下人员表示感谢。特别感谢(按姓氏字母顺序):Cecilia Baek, Philipp Kaufold, Cat Hughes, Suzanne Leijten, Michael Marcusa, Jordi Mola, Timothy Neo, Elliott Prentiss, Laia Reyes, John Ross, Julio Santil, Taylor Wilson Thomas, Mansi Tripathi, Nikhil Shanbhag, Madeleine Vos, 以及 Jackie Zajac。

