Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents 不仅要事实正确，更要溯源准确：针对 MCP 智能体的源感知验证
Our latest paper, ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents (read it on Hugging Face, or on arXiv in the meantime), targets that gap. The failure mode we care about is one we call cross-source conflation: a claim that is true somewhere in the evidence, but attributed to the wrong source. A source-blind verifier may pass it, because the fact does exist in the pool. A source-aware verifier should not.我们的最新论文《ProvenanceGuard：针对基于 MCP 的 LLM 智能体的源感知事实验证》（可在 Hugging Face 上阅读，或关注 arXiv 上的后续版本）旨在填补这一空白。我们关注的失效模式被称为“跨源混淆”：即某项主张虽然在证据池中确实存在，但被归因到了错误的来源。对于一个不具备源感知能力的验证器来说，由于事实本身存在，它可能会判定该主张通过；但一个具备源感知能力的验证器则不应如此。
The problem: supported somewhere is not the same as supported by the right source 问题所在：在某处得到支持并不等同于由正确的来源提供支持
Consider a customer support agent that answers, "According to the account record, this plan includes a 30-day refund window." The refund window may be perfectly real, but stated in a policy document, not in the account record the answer points to. Pool the two together and the claim looks supported. Keep them separate and the attribution is wrong, and in a data-sensitive setting a wrong attribution can be as damaging as a wrong fact. The same pattern shows up in a clinical agent, where a patient-specific medication detail taken from a patient-history tool becomes misleading the moment the answer presents it as a finding from the medical literature.试想一个客户支持智能体回答道：“根据账户记录，本方案包含 30 天的退款窗口。”这个退款窗口可能是真实存在的，但它是在政策文件中说明的，而非智能体所指的账户记录中。如果将两者合并在一起，该主张看起来是有据可依的。但如果将它们分开，归因就会出错；而在数据敏感的场景下，错误的归因可能与错误的事实具有同样的破坏性。同样的模式也出现在临床智能体中：当智能体将从患者病史工具中提取的特定用药细节，作为医学文献中的研究发现呈现出来时，回答便具有了误导性。
A claim can be supported by one MCP source while the answer attributes it to another. Source-blind scoring sees support in the pooled evidence and passes it; ProvenanceGuard separately checks whether the supporting source matches the one the answer states or implies. Source: paper Figure 1.一项主张可能由某个 MCP 来源提供支持，而回答却将其归因于另一个来源。源盲（Source-blind）评分机制会在汇总的证据中看到支持证据并判定其通过；而 ProvenanceGuard 会单独检查支持来源是否与回答中所陈述或暗示的来源相匹配。来源：论文图 1。
This is why faithfulness scores, useful as they are, are not enough for MCP agents. An answer carries provenance, sometimes explicitly ("according to the account record") and sometimes implicitly. ProvenanceGuard keeps that connection between claim and source available for inspection.这就是为什么忠实度分数虽然有用，但对于 MCP 智能体来说还不够。回答带有来源证明，有时是显式的（“根据账户记录”），有时是隐式的。ProvenanceGuard 将这种主张与来源之间的关联保持在可检查的状态。
What ProvenanceGuard does ProvenanceGuard 的工作原理
ProvenanceGuard is a post-generation verification layer that sits on top of a black-box MCP agent. It runs after an agent produces an answer, and never collapses the evidence into one anonymous context. Instead it carries the source identity all the way through the pipeline. It reads the captured MCP trace, including the tool outputs and their source IDs, without retraining the agent. Then it does five things in sequence: it breaks the answer into specific claims, finds the source most relevant to each one, checks whether that source actually supports it, compares the source with the one the answer names or implies, and finally emits both a per-claim source verdict and a global, answer-level allow or block decision.ProvenanceGuard 是一个位于黑盒 MCP 智能体之上的事后验证层。它在智能体生成回答后运行，绝不会将证据折叠成一个匿名的上下文。相反，它在整个流水线中始终保留来源标识。它读取捕获的 MCP 追踪记录（包括工具输出及其来源 ID），无需对智能体进行再训练。随后，它按顺序执行五项操作：将回答拆解为具体主张、为每项主张找到最相关的来源、检查该来源是否确实支持该主张、将来源与回答中陈述或暗示的来源进行比对，最后发出针对每项主张的来源判定，以及全局层面的允许或拦截决定。
The verification flow. Source identity is preserved through decomposition, routing, support scoring, attribution checking, and repair, rather than being pooled. Blocked answers can go through RARR-style repair and be re-verified. Source: paper Figure 2.验证流程。来源标识在分解、路由、支持度评分、归因检查和修复过程中始终得到保留，而不是被汇总。被拦截的回答可以经过 RARR 风格的修复并重新验证。来源：论文图 2。
A few of the design choices are worth calling out. For the experiments in our paper, we used local models so the captured traces could be processed in a controlled, offline setup: MiniLM helps find the relevant source, a DeBERTa NLI verifier model checks whether that source supports the claim, and a local language model helps break answers into claims. The verifier also checks literal values closely: a number, date, or identifier absent from the source cannot pass merely because the sentence sounds plausible. A calibrated decision step combines these signals. If an answer is blocked, a RARR-style repair step can try a source-grounded revision or a safe fallback, which the verifier then checks again.其中一些设计选择值得一提。在论文的实验中，我们使用了本地模型，以便在受控的离线设置中处理捕获的追踪记录：MiniLM 有助于查找相关来源，DeBERTa NLI 验证模型用于检查该来源是否支持该主张，本地语言模型则用于将回答拆解为各项主张。验证器还会仔细检查字面值：如果某个数字、日期或标识符在来源中不存在，即使句子听起来合理，也无法通过验证。校准后的决策步骤会综合这些信号。如果回答被拦截，RARR 风格的修复步骤会尝试进行基于来源的修订或安全的后备方案，然后由验证器再次检查。
Those named models are the setup we evaluated, not a requirement of ProvenanceGuard. The same claim, source, and decision steps can be adapted to hosted models where a team prefers cloud services; a new setup would need its own testing and calibration. Our reported results come from the local configuration. Its conservative decision policy suits data-sensitive review, where getting the source right matters more than producing the fastest possible answer.上述模型是我们评估时采用的配置，而非 ProvenanceGuard 的强制要求。同样的声明、来源和决策步骤可以适配到团队偏好的云端托管模型中；新的配置需要进行自身的测试和校准。我们报告的结果来自本地配置。其保守的决策策略适用于数据敏感的审查场景，在这种场景下，确保来源准确比生成最快的回答更为重要。
Results 结果
We tested ProvenanceGuard on answers from a medical agent that had used patient records, research articles, and other tools. This gave us 281 real traces to study. Medicine is a useful test because a fact from a patient's record and a fact from general research cannot be treated as the same source. The method can also be used in other fields when an agent keeps a record of its tool outputs and source IDs. For the main test, human experts checked 361 claims from 40 answers set aside from the data used to develop the system.我们在一个使用了患者记录、研究文章和其他工具的医疗智能体的回答上测试了 ProvenanceGuard。这为我们提供了 281 条真实追踪记录用于研究。医学是一个很好的测试领域，因为来自患者记录的事实和来自一般研究的事实不能被视为同一个来源。当智能体保留其工具输出和来源 ID 的记录时，该方法也可用于其他领域。在主要测试中，人类专家对从用于开发系统的资料中抽离出的 40 个回答中的 361 项主张进行了检查。
The most direct result is this: experts said 139 claims should not pass, and ProvenanceGuard caught 138 of them. It let one through. It also held 67 claims that the experts considered supported, sending them for review or repair. This reflects the cautious setting we tested: it favors a second look at some supported claims over letting unsupported ones through. For claims with an identifiable source, it also picked the right source about 86% of the time in this test.最直接的结果是：专家指出 139 项主张不应通过，而 ProvenanceGuard 成功捕获了其中的 138 项，仅漏掉了一项。它还拦截了 67 项专家认为有支持证据的主张，将其送交审查或修复。这反映了我们所测试的谨慎设置：它倾向于对某些有支持的主张进行二次核查，而不是让无支持的主张通过。对于具有可识别来源的主张，在此次测试中，它在大约 86% 的情况下选择了正确的来源。
We ran four other support checkers on the same claims. ProvenanceGuard scored highest on the paper's measure of how well a system catches claims that should be blocked while avoiding unnecessary blocks. The other checkers in this comparison did not tell us which tool output supported each claim. ProvenanceGuard records that connection, so a reviewer can see the source checked for each claim and the decision it produced. 我们针对相同的主张运行了其他四种支持度检查工具。ProvenanceGuard 在论文定义的指标上得分最高，该指标衡量系统在避免不必要拦截的同时捕获应拦截主张的能力。在此次对比中，其他检查工具并未告知我们每一项主张是由哪个工具输出支持的。ProvenanceGuard 记录了这种关联，因此审查员可以看到每项主张所对应的检查来源及其得出的结论。
| Verifier | Reject/block F1 | Emits claim-to-source ID |
|---|---|---|
| ProvenanceGuard (ours) | 0.802 | Yes |
| MiniCheck | 0.783 | No |
| RAGAS Faithfulness | 0.758 | No |
| AlignScore | 0.662 | No |
| SummaC-ZS | 0.436 | No |
Binary support metrics on the same held-out claim packet. ProvenanceGuard matches or beats the source-blind baselines on blocking while also producing per-claim source verdicts. Source: paper abstract and Table III.针对相同留存主张包的二元支持度指标。ProvenanceGuard 在拦截能力上持平或优于源盲基准模型，同时还能提供针对每项主张的来源判定。来源：论文摘要和表 III。
Checking claims when sources look similar 在来源相似时检查主张
In a separate, harder test with several similar sources, ProvenanceGuard scored 0.846 F1 for deciding which claims to block, but identified the exact source correctly in 50.3% of claims. Telling similar sources apart remains an important area for improvement.在另一项包含多个相似来源的难度更高的测试中，ProvenanceGuard 在决定拦截哪些主张方面获得了 0.846 的 F1 分数，但在 50.3% 的主张中准确识别出了确切来源。区分相似来源仍然是一个重要的改进领域。
We also ran a controlled test focused on wrong attribution: we changed the named source in 50 cases while leaving the supporting evidence intact. ProvenanceGuard caught all 50 swaps. This shows it can detect a clear source error, while the harder test shows the challenge of choosing among many plausible sources.我们还进行了一项针对错误归因的受控测试：我们在保持支持证据完整的情况下，更改了 50 个案例中指名的来源。ProvenanceGuard 捕获了所有 50 次错误篡改。这表明它能够检测出明确的来源错误，而难度更高的测试则显示了在多个合理来源中进行选择所面临的挑战。
Repairing blocked answers 修复被拦截的回答
Blocking is only useful if there is something to do with a blocked answer. Wired to the RARR-style repair loop, the full-trace run resolved all 173 blocked answers, though 144 of them ended in fallback text rather than a substantive rewrite, which is the system choosing to avoid an unverifiable answer rather than manufacture one. On reconstructed multi-source test traces, a fresh repair run resolved all 59 initially blocked answers with only two terminal fallbacks. As an offline gate the overhead is modest, roughly half a second per answer on the reported local configuration, with the NLI and routing calls themselves in the tens of milliseconds.拦截只有在能对被拦截的回答进行后续处理时才有用。接入 RARR 风格的修复循环后，全追踪运行解决了所有 173 个被拦截的回答，尽管其中 144 个最终以回退文本（fallback text）而非实质性重写告终，这是系统选择避免生成无法验证的回答，而不是编造一个。在重建的多源测试追踪中，一次新的修复运行解决了所有 59 个最初被拦截的回答，且仅产生了两次最终回退。作为离线门控，其开销适中，在报告的本地配置上，每个回答大约耗时半秒，其中 NLI 和路由调用的耗时仅在几十毫秒量级。
Why this fits Multiverse Computing 为什么这适合 Multiverse Computing
As agents move from single-passage RAG to multi-tool MCP setups, the question of which source a fact actually came from stops being a footnote and becomes part of what factuality means. ProvenanceGuard makes that source connection visible claim by claim. For Multiverse Computing, that means a way to check existing agents while keeping sensitive traces in a controlled environment when needed. The medical study is one use case; the same approach can be adapted wherever an agent's trace preserves its tools and sources.随着智能体从单段落 RAG 转向多工具 MCP 设置，事实究竟来自哪个来源的问题不再是脚注，而成为事实性定义的一部分。ProvenanceGuard 使得这种来源关联在每一项主张中都清晰可见。对于 Multiverse Computing 而言，这意味着在需要时，可以在受控环境中检查现有智能体，同时保持敏感追踪记录的安全。医疗研究只是其中一个用例；只要智能体的追踪记录保留了其工具和来源，同样的方法就可以适配到其他领域。
That adaptation is already visible in NVIDIA NVFlow, which merged an optional grounding-verification stage for its finance agent. It checks completed answers against the SEC excerpts the agent retrieved and saves separate decisions without changing the original rollout or training data. The NVFlow contribution uses ProvenanceGuard's source-aware verification approach; the repair loop discussed above belongs to the broader research system.这种适配已经在 NVIDIA NVFlow 中显现，它为其金融智能体合并了一个可选的接地验证（grounding-verification）阶段。它根据智能体检索到的 SEC 摘录来检查已完成的回答，并保存单独的决策，而无需更改原始部署或训练数据。NVFlow 的贡献使用了 ProvenanceGuard 的源感知验证方法；上述讨论的修复循环则属于更广泛的研究系统。
ProvenanceGuard was also presented as a poster at the Agentic AI Summit 2026 at UC Berkeley.ProvenanceGuard 也在加州大学伯克利分校举行的 2026 年智能体 AI 峰会（Agentic AI Summit 2026）上以海报形式进行了展示。
Want the full technical details, including the routing and NLI derivations, the calibration ablations, the multi-source stress slices, and the complete results tables? Read the full paper on Hugging Face, or get in touch with our team to talk about applying source-aware verification to your own agents.想要了解完整的技术细节，包括路由和 NLI 推导、校准消融实验、多源压力切片以及完整的测试结果表吗？请在 Hugging Face 上阅读完整论文，或与我们的团队联系，探讨如何将源感知验证应用到您自己的智能体中。
