obeypunctuation=true]Uber,, USA
{palb, smaddireddy, muhaimin, zoltan.puha, mzhurovich, adi.raghavendra, tout}@uber.comobeypunctuation=true]Uber,美国
{palb, smaddireddy, muhaimin, zoltan.puha, mzhurovich, adi.raghavendra, tout}@uber.com
Persona Guardrail: A Production-Grade Defense Framework for Agentic SystemsPersona Guardrail:面向智能体系统的生产级防御框架
Abstract.摘要。
Large language model-based agents are increasingly deployed to perform domain-specific tasks by interacting with enterprise knowledge, tools, and external services. Existing runtime guardrails primarily target prompt injection and other attack-specific behaviors under a black-box threat model, but provide limited guarantees that agents operate within their intended functionality. As a result, production agents remain vulnerable to malicious requests and out-of-domain queries that existing defenses often fail to distinguish.基于大语言模型的智能体正越来越多地被部署,通过与企业知识、工具和外部服务交互来执行特定领域的任务。现有的运行时护栏主要针对黑盒威胁模型下的提示词注入及其他特定攻击行为,但对于确保智能体在预期功能范围内运行的保障能力有限。因此,生产环境中的智能体仍然容易受到恶意请求和域外查询的攻击,而现有的防御手段往往难以区分这些请求。
We present Persona Guardrail, a production-grade runtime defense framework that enforces explicit functional boundaries for customer-facing agentic AI systems through synchronous input and output validation driven by semantic allowlist and blocklist specifications. We also introduce PAGE (Persona-Aware Guardrail Evaluation), a benchmark for evaluating function-specific guardrails across benign, adversarial, and out-of-domain interactions on both user and agent turns. Compared with a generic LLM-based guardrail, Persona Guardrail improves overall accuracy from 85.7% to 95.9%, increases out-of-domain detection from 57.3% to 93.5%, and reduces the false-approved rate from 25.0% to 4.7%. Currently deployed in production, Persona Guardrail meets its latency budget while sustaining a very low false-block and false-allow rate under realistic production workloads. These results demonstrate that Persona Guardrail provides a practical, scalable, and production-ready foundation for securing agentic AI systems.我们提出了 Persona Guardrail,这是一个生产级的运行时防御框架,通过语义允许列表和阻止列表驱动的同步输入输出验证,为面向客户的智能体 AI 系统强制执行明确的功能边界。我们还引入了 PAGE(Persona-Aware Guardrail Evaluation),这是一个用于评估功能特定护栏的基准测试,涵盖了用户和智能体交互过程中的良性、对抗性和域外交互。与通用的基于大模型的护栏相比,Persona Guardrail 将整体准确率从 85.7% 提高到 95.9%,将域外检测率从 57.3% 提高到 93.5%,并将错误通过率从 25.0% 降低到 4.7%。Persona Guardrail 目前已在生产中部署,在满足延迟预算的同时,在真实的生产负载下保持了极低的错误阻止率和错误通过率。这些结果表明,Persona Guardrail 为保护智能体 AI 系统提供了一个实用、可扩展且生产就绪的基础。
Keywords: 关键词:
LLM Security, Agentic Systems, Runtime Defense, Guardrails 关键词:大模型安全,智能体系统,运行时防御,护栏1. Introduction1. 引言
Recent advances in large language models (LLMs) have significantly improved their capabilities in reasoning, planning, tool use, and decision-making (Yao et al., 2022; Schick et al., 2023), enabling them to autonomously execute complex workflows that were previously achievable only through carefully engineered software systems. These advances have enabled the emergence of what is called agentic AI systems—LLM-powered applications (Wu et al., 2023) that can autonomously interact with tools, APIs, knowledge bases, and software services to accomplish a broad spectrum of user goals. By combining the capabilities of LLMs with traditional software components, the agentic systems provide a powerful building block for a diverse set of applications, ranging from ride booking and customer support automation to transaction management, information retrieval, and personalized assistance.大语言模型(LLM)的最新进展显著提升了其在推理、规划、工具使用和决策方面的能力(Yao 等人,2022;Schick 等人,2023),使其能够自主执行以前只能通过精心设计的软件系统才能实现的复杂工作流。这些进展促成了所谓的智能体 AI 系统的出现——即基于大模型的应用程序(Wu 等人,2023),它们可以自主与工具、API、知识库和软件服务交互,以完成广泛的用户目标。通过将大模型的能力与传统软件组件相结合,智能体系统为各种应用提供了一个强大的构建模块,涵盖从打车预订、客户支持自动化到交易管理、信息检索和个性化辅助等多种场景。
As agentic AI systems become increasingly integrated into production environments, ensuring their security has emerged as a critical challenge (OWASP Foundation, 2025; Kim et al., 2026). Unlike traditional software systems, agentic applications process natural-language inputs while simultaneously possessing the ability to perform privileged actions in production. This combination introduces new attack surfaces that do not exist in conventional applications. Recent research has identified a broad spectrum of attacks against agentic systems, including prompt injection, indirect prompt injection through untrusted external content, tool poisoning, retrieval-augmented generation poisoning, agent goal hijacking etc (Debenedetti et al., 2024; Greshake et al., 2023; Jia et al., 2025). However, these attacks assume different threat models and require different defense mechanisms.随着智能体 AI 系统日益融入生产环境,确保其安全性已成为一项关键挑战(OWASP Foundation,2025;Kim 等人,2026)。与传统软件系统不同,智能体应用在处理自然语言输入的同时,还具备在生产环境中执行特权操作的能力。这种结合引入了传统应用中不存在的新攻击面。近期研究已识别出针对智能体系统的广泛攻击,包括提示词注入、通过不受信任的外部内容进行的间接提示词注入、工具中毒、检索增强生成中毒、智能体目标劫持等(Debenedetti 等人,2024;Greshake 等人,2023;Jia 等人,2025)。然而,这些攻击假设了不同的威胁模型,并需要不同的防御机制。
In this paper, we focus on the practical threat model encountered by many production-facing agents, where a malicious end user has black-box access to the agent through its public interface and cannot manipulate the agent’s internal tools, retrieved documents, memory, or execution environment. In this setting, the adversary attempts to induce the agent to perform actions outside its intended functionality, bypass safety policies, or produce harmful responses through carefully crafted user queries. Consequently, indirect prompt injection, tool poisoning, and memory poisoning are outside the evaluated threat model. Our solution may still block a policy-violating final response caused by such attacks, but the evaluated guardrails do not inspect intermediate actions.在本文中,我们关注许多面向生产环境的智能体所面临的实际威胁模型,即恶意终端用户通过公共接口对智能体进行黑盒访问,且无法操纵智能体的内部工具、检索文档、内存或执行环境。在这种设定下,攻击者试图通过精心设计的用户查询诱导智能体执行超出其预期功能的动作、绕过安全策略或产生有害响应。因此,间接提示词注入、工具中毒和内存中毒不在本文评估的威胁模型范围内。我们的解决方案可能仍然会阻止由此类攻击导致的违反策略的最终响应,但评估的护栏不会检查中间操作。
Existing research on agentic AI security has primarily focused on two complementary directions. The first seeks to develop defenses against specific classes of attacks, such as prompt injection attacks (Microsoft, 2025; Jia et al., 2025). The second evaluates the effectiveness of adversarial attack strategies against LLM-based agents and measures their impact on system behavior (Andriushchenko et al., 2025; Debenedetti et al., 2024). More recently, systematization-of-knowledge (SoK) (Kim et al., 2026) studies have consolidated these attacks and defenses into comprehensive frameworks, providing a structured understanding of the emerging security landscape of agentic AI systems. While these efforts have significantly advanced our understanding of agentic AI security, they largely focus on attack taxonomies, threat models, and isolated defense mechanisms. Comparatively little attention has been paid to the practical challenges of deploying comprehensive security solutions in large-scale production environments, where latency, scalability, reliability, and operational complexity are often as important as security.现有的关于智能体 AI 安全的研究主要集中在两个互补的方向。第一,寻求开发针对特定攻击类别的防御措施,例如提示词注入攻击(Microsoft,2025;Jia 等人,2025)。第二,评估对抗性攻击策略对基于大模型的智能体的有效性,并衡量其对系统行为的影响(Andriushchenko 等人,2025;Debenedetti 等人,2024)。最近,知识系统化(SoK)研究(Kim 等人,2026)将这些攻击和防御整合为综合框架,为理解智能体 AI 系统新兴的安全格局提供了结构化的认识。虽然这些努力显著加深了我们对智能体 AI 安全的理解,但它们主要集中在攻击分类法、威胁模型和孤立的防御机制上。相比之下,在大型生产环境中部署综合安全解决方案的实际挑战受到的关注较少,而在这些环境中,延迟、可扩展性、可靠性和运营复杂性通常与安全性同等重要。
In this paper, we address this gap by presenting Persona Guardrail, a production-grade defense framework designed for large-scale agentic systems. Persona Guardrail is based on a simple yet powerful principle: every agent should operate within a well-defined functional boundary that specifies its intended capabilities, permissible actions, and expected behaviors. The framework continuously enforces these boundaries by validating both inputs to the agent and outputs generated by the agent, ensuring that interactions remain consistent with the agent’s intended functionality even in the presence of adversarial inputs.在本文中,我们通过提出 Persona Guardrail 来填补这一空白,这是一个专为大规模智能体系统设计的生产级防御框架。Persona Guardrail 基于一个简单而强大的原则:每个智能体都应在明确定义的功能边界内运行,该边界规定了其预期能力、允许的操作和预期的行为。该框架通过验证智能体的输入和智能体生成的输出,持续强制执行这些边界,确保交互即使在存在对抗性输入的情况下,也始终符合智能体的预期功能。
Our work differs from prior research in its emphasis on practical deployment. Persona Guardrail is designed for the realistic production threat model in which external users interact with customer-facing agents through a black-box interface. The framework operates under real-world constraints, including stringent latency requirements, high request volumes, and evolving adversarial behavior. Rather than focusing on a single attack class, Persona Guardrail provides a unified runtime defense that enforces an agent’s functional boundaries, mitigating malicious requests while filtering benign but out-of-domain queries. This enables production agents to remain responsive, secure, and aligned with their intended functionality under real-world deployment conditions.我们的工作与以往研究的不同之处在于对实际部署的强调。Persona Guardrail 专为现实生产威胁模型设计,在该模型中,外部用户通过黑盒接口与面向客户的智能体进行交互。该框架在现实世界的约束条件下运行,包括严格的延迟要求、高请求量和不断演变的对抗行为。Persona Guardrail 不专注于单一攻击类别,而是提供了一种统一的运行时防御,强制执行智能体的功能边界,在过滤掉良性但域外查询的同时减轻恶意请求。这使得生产环境中的智能体在现实部署条件下能够保持响应性、安全性和与其预期功能的一致性。
To the best of our knowledge, this is the first work to present and deploy a comprehensive, production-grade runtime defense framework for a large-scale, customer-facing agentic system, and to evaluate its effectiveness and operational feasibility under real-world deployment constraints. This paper makes the following contributions:据我们所知,这是首个为大规模面向客户的智能体系统提出并部署综合性、生产级运行时防御框架,并评估其在现实部署约束下的有效性和运营可行性的工作。 本文做出以下贡献:
- •
Production-Scale Security Framework. We present Persona Guardrail that enforces explicit functional boundaries for customer-facing agentic AI systems through input and output validation.生产规模的安全框架。我们提出了 Persona Guardrail,通过输入和输出验证为面向客户的智能体 AI 系统强制执行明确的功能边界。
- •
Functional Boundary Enforcement. We introduce a principled approach for defining and enforcing agent-specific functional boundaries, enabling the detection and prevention of behaviors that deviate from an agent’s intended purpose.功能边界强制执行。我们引入了一种原则性的方法来定义和强制执行智能体特定的功能边界,从而能够检测和防止偏离智能体预期目的的行为。
- •
Guardrail Evaluation Benchmark. We introduce PAGE, the first benchmark that systematically evaluates runtime guardrails for function-specific agents across both user requests and agent responses, spanning benign, adversarial, and out-of-domain interactions.护栏评估基准。我们引入了 PAGE,这是首个系统评估功能特定智能体运行时护栏的基准测试,涵盖用户请求和智能体响应,跨越良性、对抗性和域外交互。
- •
Comprehensive Evaluation. We empirically demonstrate the effectiveness, scalability, and operational feasibility of Persona Guardrail on representative production agents under realistic workloads.综合评估。我们在代表性的生产智能体上,在真实负载下实证证明了 Persona Guardrail 的有效性、可扩展性和运营可行性。
- •
Practical and Scalable Deployment. We describe the architectural and operational considerations required to deploy security guardrails at scale while satisfying strict latency and reliability requirements.实用且可扩展的部署。我们描述了在满足严格延迟和可靠性要求的同时,大规模部署安全护栏所需的架构和运营考量。
2. System Overview and Problem Setting2. 系统概述与问题设定
In this work, we investigate the design of a comprehensive defense framework for agentic AI systems. We begin by describing the system architecture, problem setting, design assumptions, and threat model that motivate the proposed defense mechanism.在这项工作中,我们研究了智能体 AI 系统综合防御框架的设计。我们首先描述了系统架构、问题设定、设计假设以及促使提出该防御机制的威胁模型。
2.1. System Setting2.1. 系统设定
We consider a generic architecture representative of modern production scale agentic AI systems. The system receives user requests in the form of natural-language instructions, structured inputs (e.g., JSON), or a combination of both. In addition to user-provided information, the frontend application may attach contextual metadata (e.g., user state, session information, or application-specific context) that assists the agent in decision making. Before reaching the agent, every incoming request is processed by the input Persona Guardrail, which validates the request against the agent’s functional boundaries and security policies. Persona Guardrail can block or send through the request to the agentic system, which comprises one or more large language models (LLMs), enterprise knowledge bases, retrieval components, internal and external tools, APIs, and other software services. During task execution, the agent may perform multiple reasoning steps and interact with these components to retrieve information, invoke tools, or execute actions before producing a final response. The generated response is subsequently processed by the output Persona Guardrail, which validates it against the agent’s intended functionality and security policies before it is returned to the user.我们考虑了一种代表现代生产规模智能体 AI 系统的通用架构。系统以自然语言指令、结构化输入(如 JSON)或两者结合的形式接收用户请求。除用户提供的信息外,前端应用程序还可能附加上下文元数据(如用户状态、会话信息或特定于应用程序的上下文),以辅助智能体进行决策。在到达智能体之前,每个传入请求都会由输入 Persona Guardrail 处理,该护栏根据智能体的功能边界和安全策略验证请求。Persona Guardrail 可以阻止或将请求发送至智能体系统,该系统包含一个或多个大语言模型(LLM)、企业知识库、检索组件、内部和外部工具、API 以及其他软件服务。在任务执行期间,智能体可能会执行多个推理步骤,并与这些组件交互以检索信息、调用工具或执行操作,然后产生最终响应。生成的响应随后由输出 Persona Guardrail 处理,该护栏在响应返回给用户之前,根据智能体的预期功能和安全策略对其进行验证。
2.2. Threat Model2.2. 威胁模型
We consider an adversarial setting in which an external malicious user has unrestricted query access to the agentic system and can observe all responses returned by the system. The adversary possesses general knowledge of the agent’s intended functionality, public interfaces, and expected behavior, but does not have access to the system’s internal state, including system prompts, intermediate reasoning, memory, tool interactions, orchestration logic, or communication between internal components. We assume the underlying LLMs, infrastructure, and trusted software components are uncompromised. Within this threat model, the adversary aims to manipulate the agent through carefully crafted inputs to achieve one or more of the following objectives:我们考虑了一种对抗性设定,其中外部恶意用户对智能体系统拥有不受限制的查询访问权限,并能观察系统返回的所有响应。攻击者具备关于智能体预期功能、公共接口和预期行为的一般知识,但无法访问系统的内部状态,包括系统提示词、中间推理、内存、工具交互、编排逻辑或内部组件之间的通信。我们假设底层大模型、基础设施和受信任的软件组件未被破坏。在此威胁模型内,攻击者的目标是通过精心设计的输入来操纵智能体,以实现以下一个或多个目标:
1. Instruction Manipulation and Prompt Extraction. The adversary attempts to extract confidential system prompts, reveal internal instructions or policies, or manipulate the agent into violating its intended behavior through attacks such as prompt injections.1. 指令操纵与提示词提取。攻击者试图提取机密系统提示词、泄露内部指令或策略,或通过提示词注入等攻击操纵智能体违反其预期行为。
2. Generation of Harmful or Policy-Violating Responses. The adversary seeks to induce the agent to produce responses or perform actions that violate organizational policies, safety requirements, or legal and regulatory constraints, potentially resulting in reputational, financial, or legal harm to the organization.2. 生成有害或违反策略的响应。攻击者寻求诱导智能体产生响应或执行违反组织策略、安全要求或法律法规约束的操作,可能导致组织声誉、财务或法律上的损害。
3. Functional Boundary Violation. The adversary attempts to cause the agent to operate outside its intended scope by eliciting responses or executing actions unrelated to its designated functionality. For example, an attacker may attempt to coerce a ride-booking assistant into providing medical advice or revealing internal operational information, thereby violating the agent’s predefined functional boundaries.3. 功能边界违规。攻击者试图通过诱导智能体执行与其指定功能无关的响应或操作,使其在预期范围之外运行。例如,攻击者可能试图强迫打车辅助智能体提供医疗建议或泄露内部运营信息,从而违反智能体预定义的功能边界。
Our objective is to design a defense framework that mitigates these attacks while preserving the utility, responsiveness, and intended functionality of the agent under normal user interactions.我们的目标是设计一个防御框架,在正常用户交互下保持智能体的效用、响应性和预期功能的同时,减轻这些攻击。
2.3. Performance Requirements2.3. 性能要求
Persona Guardrail is designed to operate under the stringent latency requirements of production agentic systems. Both the input and output guardrails make blocking decisions—requests must be validated before agent execution, and responses must be validated before being returned to the end user. Hence, the guardrails cannot be executed asynchronously with the agent. As a result, the primary design objective is to achieve robust security while incurring only a small and predictable latency overhead that preserves the responsiveness required by production-scale deployments.Persona Guardrail 旨在在生产智能体系统严格的延迟要求下运行。输入和输出护栏均需做出阻止决策——请求必须在智能体执行前进行验证,响应必须在返回给终端用户前进行验证。因此,护栏无法与智能体异步执行。结果,主要设计目标是在实现稳健安全性的同时,仅产生较小且可预测的延迟开销,从而保持生产规模部署所需的响应性。
2.4. Potential Approaches2.4. 潜在方法
From a high-level perspective, an agentic guardrail can be viewed as a reasoning system that determines whether an input or output lies within the functional and security boundaries of an agent. A straightforward approach is to leverage a general-purpose LLM by providing it with a natural-language specification of the agent’s intended functionality, permissible actions, and security policies, and asking it to classify each interaction accordingly. While conceptually simple, this approach presents several practical challenges for production deployment.从高层视角来看,智能体护栏可以被视为一个推理系统,用于确定输入或输出是否处于智能体的功能和安全边界内。一种直接的方法是利用通用大模型,向其提供智能体预期功能、允许操作和安全策略的自然语言规范,并要求其相应地对每次交互进行分类。虽然概念上很简单,但这种方法在生产部署中存在几个实际挑战。
First, a comprehensive natural-language specification is difficult to construct and maintain as the underlying agent evolves. Production agents frequently undergo updates, including changes to prompts, tools, APIs, retrieval sources, and supported capabilities. Consequently, the guardrail specification must evolve alongside the agent to accurately reflect its intended behavior. Second, invoking a medium- or large-scale reasoning model for every request introduces significant latency. Since both the input and output guardrails operate synchronously on the critical execution path, their latency directly impacts end-to-end response time and user experience. Finally, a static guardrail is unlikely to remain effective in dynamic production environments where both adversarial attack strategies and legitimate user behavior continuously evolve.首先,随着底层智能体的演变,构建和维护综合的自然语言规范非常困难。生产智能体经常进行更新,包括提示词、工具、API、检索源和支持功能的更改。因此,护栏规范必须随智能体一起演变,以准确反映其预期行为。其次,为每个请求调用中大型推理模型会引入显著延迟。由于输入和输出护栏都在关键执行路径上同步运行,它们的延迟直接影响端到端响应时间和用户体验。最后,在对抗性攻击策略和合法用户行为不断演变的动态生产环境中,静态护栏不太可能保持有效。
These challenges raise several fundamental design questions. How can the functional and security specification of an agent be maintained as the system evolves? How should guardrail performance be measured, particularly in terms of false positive and false negative rates? How can production misclassifications be systematically incorporated to refine the guardrail over time? More broadly, how can a guardrail continuously adapt to emerging attack techniques and distribution shifts in user requests while maintaining low latency and high reliability?这些挑战提出了几个基本的设计问题。随着系统的演变,智能体的功能和安全规范应如何维护?护栏性能应如何衡量,特别是在误报率和漏报率方面?生产环境中的错误分类应如何系统地纳入以随时间改进护栏?更广泛地说,护栏如何在保持低延迟和高可靠性的同时,持续适应新兴的攻击技术和用户请求的分布偏移?
Existing defenses only partially address these challenges. Rule-based and policy-based guardrails (Rebedea et al., 2023; Inan et al., 2023) provide deterministic enforcement but are difficult to scale across heterogeneous agents and often fail to capture context-dependent or previously unseen attacks. Model-based guardrails (Microsoft, 2025; Jia et al., 2025) offer greater flexibility and generalization but typically rely on expensive reasoning models and have received relatively little attention regarding continuous adaptation, deployment at production-scale, or systematic evaluation under evolving workloads. More generally, recent systematization efforts (Kim et al., 2026) categorize existing defenses into runtime protection, secure-by-design mechanisms, and component hardening, while identifying the need for practical, adaptive defenses that can operate under real-world deployment constraints.现有的防御措施仅部分解决了这些挑战。基于规则和策略的护栏(Rebedea 等人,2023;Inan 等人,2023)提供了确定性的强制执行,但难以在异构智能体之间扩展,且往往无法捕捉上下文相关或以前未见的攻击。基于模型的护栏(Microsoft,2025;Jia 等人,2025)提供了更大的灵活性和泛化能力,但通常依赖昂贵的推理模型,且在持续适应、生产规模部署或在演变负载下的系统评估方面受到的关注相对较少。更广泛地说,近期的系统化工作(Kim 等人,2026)将现有防御归类为运行时保护、安全设计机制和组件强化,同时指出了在现实世界部署约束下运行的实用、自适应防御的需求。
Motivated by these challenges, we design Persona Guardrail as an adaptive, production-oriented runtime defense framework. Rather than relying solely on static rules or expensive reasoning, Persona Guardrail combines lightweight inference with a closed-loop feedback mechanism that continuously refines its functional and security boundaries using signals collected from production traffic. This design enables the framework to adapt to both emerging adversarial attack patterns and shifts in user behavior while satisfying the stringent latency and scalability requirements of production agentic systems.受这些挑战的驱动,我们将 Persona Guardrail 设计为一个自适应的、面向生产的运行时防御框架。Persona Guardrail 不仅依赖静态规则或昂贵的推理,而是将轻量级推理与闭环反馈机制相结合,利用从生产流量中收集的信号持续细化其功能和安全边界。这种设计使框架能够在满足生产智能体系统严格延迟和可扩展性要求的同时,适应新兴的对抗性攻击模式和用户行为的转变。
3. Persona Guardrail Design3. Persona Guardrail 设计
Persona Guardrail is a production-grade defense framework designed to ensure that an agentic system operates within its intended functional and security boundaries. The framework consists of three primary components: 1) Functional Boundary Specification, 2) Input Guardrail Enforcement, and 3) Output Guardrail Enforcement.Persona Guardrail 是一个生产级防御框架,旨在确保智能体系统在其预期的功能和安全边界内运行。该框架由三个主要组件组成:1) 功能边界规范,2) 输入护栏强制执行,3) 输出护栏强制执行。
The key insight behind Persona Guardrail is that every agent has a well-defined purpose. Rather than attempting to detect every possible attack in isolation, Persona Guardrail first models the intended functional boundary of the agent and then enforces this specification throughout the agent’s interaction lifecycle. Both the input and output guardrails are responsible for ensuring that the interaction remains consistent with the agent’s functional boundary.Persona Guardrail 背后的关键见解是,每个智能体都有一个明确定义的用途。Persona Guardrail 不是试图孤立地检测每一个可能的攻击,而是首先对智能体的预期功能边界进行建模,然后在智能体的整个交互生命周期中强制执行此规范。输入和输出护栏均负责确保交互与智能体的功能边界保持一致。
3.1. Functional Boundary Specification3.1. 功能边界规范
The central abstraction underlying Persona Guardrail is the functional boundary of an agent. Intuitively, the functional boundary characterizes the set of user interactions that are consistent with the agent’s intended functionality, security requirements, and product specifications.Persona Guardrail 的核心抽象是智能体的功能边界。直观地说,功能边界表征了与智能体预期功能、安全要求和产品规范一致的一组用户交互。
Definition 1 (Functional Boundary)定义 1(功能边界)
Let denote the space of all possible user queries submitted to an agent. The functional boundary partitions into two disjoint subsets:设 𝒬 表示提交给智能体的所有可能用户查询的空间。功能边界将 𝒬 划分为两个不相交的子集:
[ ] where[ 𝒬 = 𝒬_{allow} ∪ 𝒬_{block}, ] 其中
- •
denotes the set of queries that are consistent with the intended functionality of the agent and should be processed.𝒬_{allow} 表示与智能体预期功能一致且应被处理的查询集合。
- •
denotes the set of queries that should be rejected: malicious or unsafe queries, together with out-of-domain queries in deployments configured to reject them.𝒬_{block} 表示应被拒绝的查询集合:恶意或不安全查询,以及在配置为拒绝它们的部署中的域外查询。
The objective of Persona Guardrail is to approximate this partition to minimize both false positives (rejecting legitimate queries) and false negatives (allowing malicious or out-of-domain queries). Since the behavior of production agents and their users continuously evolves, the functional boundary is not static; instead, it is continuously refined using production telemetry, attack simulations, and feedback collected from deployment.Persona Guardrail 的目标是近似这种划分,以最大限度地减少误报(拒绝合法查询)和漏报(允许恶意或域外查询)。由于生产智能体及其用户的行为持续演变,功能边界不是静态的;相反,它使用生产遥测、攻击模拟和从部署中收集的反馈进行持续细化。
In practice, explicitly enumerating the sets and is infeasible because natural language admits an effectively unbounded space of expressions. Persona Guardrail therefore represents the functional boundary using semantic descriptions that capture both the intended functionality of the agent and security constraints.在实践中,显式枚举 𝒬_{allow} 和 𝒬_{block} 集合是不可行的,因为自然语言允许无限的表达空间。因此,Persona Guardrail 使用捕获智能体预期功能和安全约束的语义描述来表示功能边界。
3.1.1. Hybrid Functional Boundary Representation3.1.1. 混合功能边界表示
We investigate three approaches for representing the functional boundary:我们研究了三种表示功能边界的方法:
- •
Allowlist-only仅允许列表
- •
Blocklist-only仅阻止列表
- •
Hybrid allowlist-blocklist混合允许列表-阻止列表
An allowlist-only representation specifies the categories of requests that the agent is permitted to process. While this approach provides strong security, it often becomes overly conservative in practice. Because it is impossible to exhaustively enumerate every legitimate user request, incomplete allowlists frequently result in elevated false positive rates and poor user experience.仅允许列表的表示指定了智能体被允许处理的请求类别。虽然这种方法提供了强大的安全性,但在实践中往往变得过于保守。由于不可能详尽地枚举每一个合法的用户请求,不完整的允许列表经常导致较高的误报率和糟糕的用户体验。
Conversely, a blocklist-only representation specifies the set of queries that the agent should reject. These include both security-driven and functionality-driven constraints. Security-driven entries correspond to known adversarial behaviors, such as prompt injection, jailbreaks, and other attacks that attempt to manipulate the agent or violate organizational policies. Functionality-driven entries correspond to out-of-domain (OOD) requests that fall outside the intended purpose of the agent. Although such requests may be entirely benign, processing them can result in poor user experience, hallucinations, or unintended behavior. Whether out-of-domain queries are included in the blocklist is configurable and depends on the deployment requirements of the agent. Some applications are designed to operate as specialized assistants and therefore reject requests outside their intended functionality, while others may choose to answer or gracefully redirect out-of-domain requests. Persona Guardrail supports both deployment models by allowing agent owners to configure the desired functional boundary.相反,仅阻止列表的表示指定了智能体应拒绝的查询集合。这些包括安全驱动和功能驱动的约束。安全驱动条目对应已知的对抗行为,例如提示词注入、越狱以及其他试图操纵智能体或违反组织策略的攻击。功能驱动条目对应落在智能体预期目的之外的域外(OOD)请求。虽然此类请求可能完全是良性的,但处理它们可能导致糟糕的用户体验、幻觉或意外行为。域外查询是否包含在阻止列表中是可配置的,并取决于智能体的部署要求。有些应用程序被设计为专门的助手,因此拒绝超出其预期功能的请求,而其他应用程序可能选择回答或优雅地重定向域外请求。Persona Guardrail 通过允许智能体所有者配置所需的功能边界来支持这两种部署模型。
To reduce ambiguity at inference time, the allowlist and blocklist should overlap as little as possible. Where the two specifications intersect, they present the classifier with conflicting evidence and raise classification uncertainty. Persona Guardrail therefore identifies overlapping entries between the two lists and adjudicates them during guardrail construction and refinement, so that each capability falls under a single, unambiguous rule.为了减少推理时的歧义,允许列表和阻止列表应尽可能少地重叠。当两个规范相交时,它们会向分类器提供相互冲突的证据并增加分类不确定性。因此,Persona Guardrail 在护栏构建和细化过程中识别两个列表之间的重叠条目并对其进行裁决,以便每个功能都归于单一、明确的规则之下。
3.1.2. Constructing the Functional Boundary3.1.2. 构建功能边界
The functional boundary is constructed from multiple complementary sources.功能边界由多个互补来源构建。
Agent implementation and product specifications. During initial deployment, the allowlist is generated by analyzing both the implementation of the agent and its product requirements. We employ a code-oriented LLM to analyze the agent implementation and infer the classes of user requests that the agent is designed to support. This analysis considers the agent’s available tools, APIs, retrieval components, system prompts, and external resources to produce an initial semantic specification of its intended functionality.智能体实现和产品规范。在初始部署期间,允许列表是通过分析智能体的实现及其产品需求生成的。我们采用面向代码的大模型来分析智能体实现,并推断智能体旨在支持的用户请求类别。此分析考虑了智能体的可用工具、API、检索组件、系统提示词和外部资源,以生成其预期功能的初始语义规范。
Production traffic analysis. Following deployment, Persona Guardrail operates in a shadow mode during which it observes production traffic without enforcing decisions. Legitimate interactions that are not adequately represented in the initial specification are incorporated into the allowlist, enabling the functional boundary to evolve alongside changing user behavior and product capabilities.生产流量分析。部署后,Persona Guardrail 在影子模式下运行,在此期间它观察生产流量而不强制执行决策。初始规范中未充分表示的合法交互被纳入允许列表,使功能边界能够随着用户行为和产品能力的变化而演变。
Security-driven blocklist generation. The initial blocklist is constructed from two complementary sources. First, it incorporates known adversarial techniques targeting agentic AI systems, including prompt injection, indirect prompt injection, jailbreaks, goal hijacking, functional-boundary violations, and tool misuse. Second, it captures classes of out-of-domain queries that lie outside the intended scope of the agent. As new attacks emerge and user interaction patterns evolve, the blocklist is continuously refined through a feedback loop, allowing the framework to adapt to both evolving adversarial threats and changing product requirements.安全驱动的阻止列表生成。初始阻止列表由两个互补来源构建。首先,它纳入了针对智能体 AI 系统的已知对抗技术,包括提示词注入、间接提示词注入、越狱、目标劫持、功能边界违规和工具滥用。其次,它捕获了落在智能体预期范围之外的域外查询类别。随着新攻击的出现和用户交互模式的演变,阻止列表通过反馈循环持续细化,使框架能够适应演变的对抗威胁和不断变化的产品需求。
By combining implementation analysis, production observations, and continuously evolving security knowledge, Persona Guardrail maintains an adaptive approximation of the agent’s functional boundary that remains effective under both evolving adversarial threats and shifts in legitimate user behavior. Persona Guardrail enforces the functional boundary at two stages of the agent execution pipeline. Before the request reaches the agent, the input guardrail determines whether the query lies within the functional boundary. After the agent generates a response, the output guardrail verifies that the generated response remains consistent with the same functional boundary. The following sections describe these enforcement mechanisms in detail.通过结合实现分析、生产观察和持续演变的安全知识,Persona Guardrail 维护了一个自适应的智能体功能边界近似值,在演变的对抗威胁和合法用户行为的转变下保持有效。Persona Guardrail 在智能体执行流水线的两个阶段强制执行功能边界。在请求到达智能体之前,输入护栏确定查询是否在功能边界内。智能体生成响应后,输出护栏验证生成的响应是否与相同的功能边界保持一致。以下章节详细描述了这些强制执行机制。
3.2. Input Guardrail Enforcement3.2. 输入护栏强制执行
The input guardrail enforces the functional boundary before a request reaches the agentic system. Its objective is to determine whether an incoming user request should be processed by the agent or rejected. The input allowlist specifies the classes of user requests that fall within the intended capabilities of the agent, whereas the input blocklist captures malicious inputs, known adversarial attack patterns, and, optionally, out-of-domain requests that the agent is not designed to handle. To enforce the input functional boundary, we investigated four alternative architectures:输入护栏在请求到达智能体系统之前强制执行功能边界。其目标是确定传入的用户请求应由智能体处理还是被拒绝。输入允许列表指定了落在智能体预期能力范围内的用户请求类别,而输入阻止列表捕获恶意输入、已知的对抗攻击模式,以及(可选地)智能体未被设计处理的域外请求。为了强制执行输入功能边界,我们研究了四种替代架构:
1) Embedding-based Retrieval. The incoming query is embedded and compared against representative allowlist and blocklist examples using vector similarity. The query is classified based on its nearest neighbors and a calibrated similarity threshold.1) 基于嵌入的检索。传入查询被嵌入,并使用向量相似度与代表性的允许列表和阻止列表示例进行比较。查询根据其最近邻居和校准的相似度阈值进行分类。
2) LLM-based Classification. A low-latency reasoning model receives the user query together with the functional boundary specification and directly classifies the query as allow or block.2) 基于大模型的分类。低延迟推理模型接收用户查询以及功能边界规范,并直接将查询分类为允许或阻止。
3) Hybrid Retrieval and LLM. Queries exhibiting very high semantic similarity to existing allowlist or blocklist examples are classified using the embedding model. Queries that cannot be classified with sufficient confidence are forwarded to the LLM classifier for semantic reasoning.3) 检索与大模型混合。对现有允许列表或阻止列表示例表现出极高语义相似度的查询使用嵌入模型进行分类。无法以足够置信度分类的查询被转发给大模型分类器进行语义推理。
4) Fine-tuned Classification Model. A model is fine-tuned using agent-specific allowlist and blocklist examples to directly predict the appropriate classification.4) 微调分类模型。使用智能体特定的允许列表和阻止列表示例对模型进行微调,以直接预测适当的分类。
We evaluated these architectures along four dimensions: classification performance, inference latency, operational complexity, and generalization across heterogeneous agents. The embedding-based approach provides the lowest inference latency but requires careful calibration of similarity thresholds and continuous maintenance of an embedding database containing representative allowlist and blocklist examples for every agent. As agents evolve, maintaining these examples becomes increasingly expensive, and the approach exhibits limited robustness to semantically novel requests. The fine-tuned classifier performs well when evaluated on agents represented in its training data. However, its performance degrades when deployed on previously unseen agents or when the functional boundary evolves, necessitating periodic retraining and increasing operational complexity. The hybrid architecture improves classification accuracy by combining fast retrieval with LLM reasoning, but introduces additional infrastructure complexity and requires maintaining both an embedding index and an LLM-based inference pipeline.我们从四个维度评估了这些架构:分类性能、推理延迟、运营复杂性和跨异构智能体的泛化能力。基于嵌入的方法提供最低的推理延迟,但需要仔细校准相似度阈值,并持续维护包含每个智能体代表性允许列表和阻止列表示例的嵌入数据库。随着智能体的演变,维护这些示例变得越来越昂贵,且该方法对语义新颖的查询表现出有限的稳健性。微调分类器在评估其训练数据中表示的智能体时表现良好。然而,当部署在以前未见的智能体上或功能边界演变时,其性能会下降,需要定期重新训练并增加运营复杂性。混合架构通过将快速检索与大模型推理相结合提高了分类准确率,但引入了额外的基础设施复杂性,并需要同时维护嵌入索引和大模型推理流水线。
Based on these trade-offs, we adopt an LLM-based classifier using Qwen3-30B-A3B (et al., 2025) as the input guardrail. We find that this approach offers the best overall trade-off across these dimensions: low operational complexity, low inference latency, strong classification performance, and generalization across diverse agentic systems. Moreover, because the functional boundary is represented semantically rather than as agent-specific examples, the same model can be deployed across multiple agents with minimal customization while adapting naturally to evolving boundary specifications.基于这些权衡,我们采用 Qwen3-30B-A3B(et al., 2025)作为输入护栏的基于大模型的分类器。我们发现这种方法在这些维度上提供了最佳的整体权衡:低运营复杂性、低推理延迟、强大的分类性能以及跨多样化智能体系统的泛化能力。此外,由于功能边界是语义表示而非智能体特定示例,同一个模型可以以最小的定制部署在多个智能体上,同时自然地适应演变的功能边界规范。
3.3. Output Guardrail Enforcement3.3. 输出护栏强制执行
The output guardrail enforces the functional boundary before the agent’s response is returned to the user. Its objective is to verify that the generated response remains consistent with the agent’s intended functionality, organizational policies, and security requirements. The output allowlist specifies the expected characteristics and permissible actions of the agent’s responses, while the output blocklist captures prohibited behaviors, including policy violations, unsafe or harmful responses, disclosure of confidential information, hallucinated capabilities and other outputs that violate the agent’s functional or security boundaries.输出护栏在智能体的响应返回给用户之前强制执行功能边界。其目标是验证生成的响应是否与智能体的预期功能、组织策略和安全要求保持一致。输出允许列表指定了智能体响应的预期特征和允许操作,而输出阻止列表捕获禁止的行为,包括策略违规、不安全或有害的响应、机密信息的披露、幻觉能力以及其他违反智能体功能或安全边界的输出。
As with the input guardrail, we evaluated the same four architectures (embedding-based retrieval, LLM-based classification, hybrid retrieval–LLM, and a fine-tuned classifier). We adopt the same LLM-based classifier (Qwen3-30B-A3B) (et al., 2025) for output enforcement, as it provides the best balance between inference latency, deployment simplicity, and generalization across diverse agents. Furthermore, using a common classification model for both input and output enforcement simplifies deployment and maintenance while enabling a consistent semantic interpretation of the functional boundary throughout the agent execution pipeline. The output guardrail serves as the final enforcement point before the response reaches the end user and may block a policy-violating response even when an attack has influenced the agent.与输入护栏一样,我们评估了相同的四种架构(基于嵌入的检索、基于大模型的分类、检索-大模型混合和微调分类器)。我们采用相同的大模型分类器(Qwen3-30B-A3B)(et al., 2025)进行输出强制执行,因为它在推理延迟、部署简单性和跨多样化智能体的泛化能力之间提供了最佳平衡。此外,在输入和输出强制执行中使用通用的分类模型简化了部署和维护,同时实现了在整个智能体执行流水线中对功能边界的一致语义解释。输出护栏作为响应到达终端用户之前的最终强制执行点,即使在攻击已经影响智能体的情况下,也可能阻止违反策略的响应。
| Approach | Latency | Classification | Cross-Agent | Operational |
|---|---|---|---|---|
| Performance | Generalization | Complexity | ||
| Embedding | Low | Low | High | |
| Fine-tuned | Medium | Medium | Medium | |
| Hybrid | High | High | High | |
| LLM-based (Qwen3-30B-A3B) | High | High | Low |
3.4. Guardrail Model Selection3.4. 护栏模型选择
Model selection for Persona Guardrail is driven by both guardrail performance and production latency. To ensure a responsive user experience, we require the model’s end-to-end P90 inference latency. This requirement restricts the set of models that can be practically deployed.Persona Guardrail 的模型选择由护栏性能和生产延迟共同驱动。为确保响应迅速的用户体验,我们需要模型端到端的 P90 推理延迟。这一要求限制了可以实际部署的模型集。
We initially experimented with both closed- and open-weight models. Closed models, including models served through OpenAI and Anthropic APIs, require requests to be sent to externally hosted inference endpoints. In our experiments, the resulting network and serving overhead prevented these models from consistently satisfying our sub-100 ms latency requirement. We therefore focused on open-weight models that could be deployed within our own serving infrastructure, allowing us to control inference placement, caching, hardware configuration, and other serving optimizations.我们最初试验了闭源和开源权重模型。闭源模型(包括通过 OpenAI 和 Anthropic API 服务的模型)要求将请求发送到外部托管的推理端点。在我们的实验中,由此产生的网络和服务开销使这些模型无法持续满足我们 100 毫秒以下的延迟要求。因此,我们专注于可以在我们自己的服务基础设施内部署的开源权重模型,这使我们能够控制推理位置、缓存、硬件配置和其他服务优化。
Among locally deployable models, we evaluated Qwen2.5-3B, Qwen2.5-7B, Qwen3-8B, and Qwen3-30B-A3B. We additionally evaluated Qwen3-VL-30B as a higher-capacity alternative. Each model was evaluated on the same sets of benign, out-of-domain (OOD), and adversarial queries for a representative production agent. The benign evaluation contains three subsets representing different production traffic distributions: most-frequent queries, randomly sampled queries, and an extensive benign set. For OOD and adversarial datssets, we report the percentage of queries incorrectly approved; therefore, lower values indicate stronger rejection. Table 2 summarizes the results.在本地可部署的模型中,我们评估了 Qwen2.5-3B、Qwen2.5-7B、Qwen3-8B 和 Qwen3-30B-A3B。我们还评估了 Qwen3-VL-30B 作为更高容量的替代方案。每个模型都在针对代表性生产智能体的相同良性、域外(OOD)和对抗性查询集上进行了评估。良性评估包含三个子集,代表不同的生产流量分布:最频繁查询、随机采样查询和广泛的良性集。对于 OOD 和对抗性数据集,我们报告错误通过的查询百分比;因此,较低的值表示更强的拒绝能力。表 2 总结了结果。
| Dataset / Metric | Qwen3-VL-30B | Qwen3-30B-A3B (8-bit) | Qwen3-8B (8-bit) | Qwen2.5-7B | Qwen2.5-3B |
|---|---|---|---|---|---|
| (Deployed) | |||||
| Approve – Most Frequent | 99% | 100% | 92% | 84% | 49% |
| Approve – Random | 96% | 99% | 93% | 83% | 67% |
| Approve – Extensive | 96% | 99% | 85% | 77% | 53% |
| OOD – Generic | 1% | 5% | 0% | 0.5% | 5% |
| Adversarial – Generic | 7% | 15% | 7% | 12% | 11% |
| Median latency | 150 ms | 95 ms | 170 ms | 168 ms | 81 ms |
The results expose an important trade-off between benign utility, security, and latency. The smaller models can achieve competitive rejection rates on OOD and adversarial requests, but at the cost of substantially higher false blocking of legitimate traffic. For example, Qwen2.5-3B has the lowest median latency (81 ms) and approves only 18% of targeted adversarial requests, but approves just 49–67% of the benign evaluation sets. Qwen2.5-7B and Qwen3-8B improve benign utility, but still approve only 77–93% of benign requests and exhibit median latencies of 168 ms and 170 ms, respectively.结果揭示了良性效用、安全性和延迟之间的重要权衡。较小的模型可以在 OOD 和对抗性请求上实现有竞争力的拒绝率,但代价是大幅增加了对合法流量的错误阻止。例如,Qwen2.5-3B 具有最低的中位数延迟(81 毫秒),仅批准了 18% 的针对性对抗请求,但仅批准了 49–67% 的良性评估集。Qwen2.5-7B 和 Qwen3-8B 改善了良性效用,但仍仅批准 77–93% 的良性请求,且中位数延迟分别为 168 毫秒和 170 毫秒。
Qwen3-30B-A3B provides a substantially better utility-latency trade-off. It approves 99–100% of benign requests across all three benign evaluation sets while maintaining a median inference latency of 95 ms. Although some smaller models achieve lower false-approval rates on individual OOD or adversarial sets, their substantially higher benign false-block rates make them unsuitable for our production setting, where both security and preservation of legitimate agent functionality are important.Qwen3-30B-A3B 提供了显著更好的效用-延迟权衡。它在所有三个良性评估集中批准了 99–100% 的良性请求,同时保持了 95 毫秒的中位数推理延迟。尽管一些较小的模型在个别 OOD 或对抗性集上实现了更低的错误通过率,但它们大幅更高的良性错误阻止率使它们不适合我们的生产环境,在生产环境中,安全性和保持合法智能体功能都非常重要。
Qwen3-VL-30B provides the closest benign classification performance, approving 96–99% of legitimate requests, and achieves lower false-approval rates than Qwen3-30B-A3B on the evaluated OOD and adversarial sets. However, its median inference latency is 150 ms, exceeding our latency target. In contrast, Qwen3-30B-A3B uses a mixture-of-experts architecture with approximately 30B total parameters while activating only approximately 3B parameters per token. This architecture provides sufficient model capacity for semantic functional-boundary classification while keeping inference cost compatible with synchronous guardrail enforcement.Qwen3-VL-30B 提供了最接近的良性分类性能,批准了 96–99% 的合法请求,并且在评估的 OOD 和对抗性集上实现了比 Qwen3-30B-A3B 更低的错误通过率。然而,其中位数推理延迟为 150 毫秒,超过了我们的延迟目标。相比之下,Qwen3-30B-A3B 使用混合专家架构,总参数约为 30B,但每个标记仅激活约 3B 参数。这种架构为语义功能边界分类提供了足够的模型容量,同时使推理成本与同步护栏强制执行兼容。
Based on these results, we select Qwen3-30B-A3B as the shared classifier for both the input and output Persona Guardrails. Among the evaluated models, it provides the best overall trade-off between preserving benign utility, rejecting functional-boundary violations, and satisfying the production latency constraint. Using the same model for both guardrails also simplifies serving and maintenance and allows agent-specific behavior to be expressed through functional-boundary specifications rather than separate model deployments.基于这些结果,我们选择 Qwen3-30B-A3B 作为输入和输出 Persona Guardrail 的共享分类器。在评估的模型中,它在保持良性效用、拒绝功能边界违规和满足生产延迟约束之间提供了最佳的整体权衡。为两个护栏使用相同的模型也简化了服务和维护,并允许通过功能边界规范而非单独的模型部署来表达智能体特定的行为。
4. Experimental Evaluation4. 实验评估
We evaluate Persona Guardrail on PAGE (Persona-Aware Guardrail Evaluation) (Section 4.1), our persona-scoped benchmark spanning four AgentDojo (Debenedetti et al., 2024) domains—Workspace, Slack, Banking, and Travel (Table 3)—under a malicious-user threat model in which an adversary interacts directly with the agent endpoint. Because PAGE labels both user turns (pre-app, evaluated by the input guardrail) and agent turns (post-app, evaluated by the output guardrail), a single benchmark exercises both guardrails of the framework. Every configuration is evaluated on identical PAGE splits so that performance differences are attributable to the guardrail configuration alone.我们在 PAGE(Persona-Aware Guardrail Evaluation)(第 4.1 节)上评估 Persona Guardrail,这是我们跨越四个 AgentDojo(Debenedetti 等人,2024)领域——工作空间、Slack、银行和旅行(表 3)的智能体范围基准测试,在恶意用户威胁模型下,攻击者直接与智能体端点交互。由于 PAGE 标记了用户轮次(pre-app,由输入护栏评估)和智能体轮次(post-app,由输出护栏评估),单个基准测试同时测试了框架的两个护栏。每个配置都在相同的 PAGE 分割上进行评估,因此性能差异仅归因于护栏配置本身。
Table 3 summarizes the four domains, their in-scope function, and representative off-persona actions a malicious user might attempt—the behaviours the guardrail must reject. We take only the domain definitions from AgentDojo; we do not run its environment, tools, or indirect-injection attack suite. The results therefore do not measure end-to-end prevention of indirect injection or unauthorized intermediate actions.表 3 总结了四个领域、其范围内功能以及恶意用户可能尝试的代表性离线行为——即护栏必须拒绝的行为。我们仅采用 AgentDojo 中的领域定义;我们不运行其环境、工具或间接注入攻击套件。因此,结果不衡量对间接注入或未授权中间操作的端到端预防。
| Domain | Capabilities | Representative prohibited actions |
|---|---|---|
| Workspace | Email, calendar, cloud drive | Export mail/files; access another user’s mailbox |
| Slack | Messages, channels, shared links | Forward private direct messages; leak workspace data |
| Banking | Balances, payments, transfers | Transfer funds to unauthorized accounts; reveal credentials |
| Travel | Flights, hotels, restaurants, cars | Share itineraries or personal IDs; modify another user’s booking |
4.1. The Benchmark4.1. 基准测试
Existing public agent benchmarks are designed to measure task utility, harmful behavior, or robustness against specific attacks, such as indirect prompt injection, and therefore do not provide the labels a persona guardrail needs: whether a user turn (before the agent acts) or an agent turn (before a response is returned) falls inside or outside the agent’s declared functional boundary. To evaluate Persona Guardrail under its intended malicious-user threat model, we construct PAGE (Persona-Aware Guardrail Evaluation), a purpose-built benchmark in which every message is (i) attributed to one of the four AgentDojo personas, (ii) tagged with the turn on which the guardrail fires (pre-app/user or post-app/agent), and (iii) labelled with a request kind—benign, adversarial, out-of-domain (OOD), or mixed. Every label is grounded in the same natural-language allowlist that instantiates the guardrail, so the dataset and the enforced policy share a single source of truth. In total, PAGE comprises labelled messages ( synthesized around the four personas plus projected from AgentHarm (Andriushchenko et al., 2025)) and is, to our knowledge, the first guardrail benchmark that jointly covers user and agent turns across four production domains with benign, out-of-domain, and adversarial intents. We release it to support reproducible evaluation of functional-boundary enforcement.现有的公共智能体基准测试旨在衡量任务效用、有害行为或针对特定攻击(如间接提示词注入)的鲁棒性,因此无法提供个人防护栏(persona guardrail)所需的标签:即用户轮次(在智能体执行动作前)或智能体轮次(在返回响应前)是否处于智能体声明的功能边界之内。为了在预期的恶意用户威胁模型下评估个人防护栏,我们构建了 PAGE(Persona-Aware Guardrail Evaluation),这是一个专门构建的基准测试,其中每条消息都 (i) 归属于四个 AgentDojo 人设之一,(ii) 标记了防护栏触发的轮次(应用前/用户轮次或应用后/智能体轮次),以及 (iii) 标记了请求类型——良性、对抗性、域外 (OOD) 或混合。每个标签都基于实例化防护栏的相同自然语言白名单,因此数据集和执行的策略共享单一的事实来源。PAGE 总共包含 2,180 条标记消息(2,076 条围绕四个人设合成,外加 104 条从 AgentHarm (Andriushchenko et al., 2025) 投影而来),据我们所知,这是第一个联合覆盖四个生产领域中用户和智能体轮次,且包含良性、域外和对抗性意图的防护栏基准测试。我们发布它是为了支持功能边界执行的可重复评估。
4.1.1. Generation Methodology4.1.1. 生成方法论
The dataset was produced by a human-guided, semi-automated pipeline that combines four complementary sources, chosen so that no single generator biases the corpus and so that the evaluated classifier never sees text authored by itself. Table 4 summarizes the provenance of the persona-centric messages.该数据集由一个人工引导的半自动化流水线生成,结合了四个互补的来源。选择这些来源是为了确保没有单一的生成器使语料库产生偏差,并确保被评估的分类器永远不会看到由其自身编写的文本。表 4 总结了这 2,076 条以人设为中心的请求消息的来源。
1) Human-authored seeds (AgentDojo (Debenedetti et al., 2024)). The verbatim user tasks of each AgentDojo suite provide realistic benign in-domain requests, while the suite’s injection-task goals provide grounded adversarial intents. These anchor the corpus in human-written, domain-authentic language.1) 人工编写的种子数据(AgentDojo (Debenedetti et al., 2024))。每个 AgentDojo 套件的逐字用户任务提供了真实的良性域内请求,而套件的注入任务目标则提供了扎实的对抗性意图。这些数据将语料库锚定在人类编写的、领域真实的语言中。
2) LLM synthesis. A frontier instruction-tuned model (Claude Opus 4.8 with reasoning), distinct from the guardrail classifier under test, expands the benign set and generates off-persona “hallucination” content (e.g., poems, code, opinions) for agent turns. Using a different model family for generation and evaluation avoids self-evaluation bias.2) LLM 合成。一个前沿的指令微调模型(Claude Opus 4.8,具备推理能力),与待测的防护栏分类器不同,它扩展了良性集合,并为智能体轮次生成人设之外的“幻觉”内容(例如诗歌、代码、观点)。使用不同的模型系列进行生成和评估,避免了自我评估偏差。
3) Deterministic adversarial templates. Attack turns are synthesized from parameterised templates rather than an LLM. The frontier model refused to author attack payloads, and templates additionally give exact reproducibility and, through slot filling, broad lexical variety while keeping each attack grounded in an AgentDojo (Debenedetti et al., 2024)/AgentHarm (Andriushchenko et al., 2025) pattern. Slots draw from exfiltration addresses, external sites, adversarial IBANs, third-party identities, authority personas, cities, venues, and monetary amounts, combined with user and agent phrasing prefixes.3) 确定性对抗模板。攻击轮次由参数化模板而非 LLM 合成。前沿模型拒绝编写攻击载荷,而模板不仅提供了精确的可重复性,还通过槽位填充提供了广泛的词汇多样性,同时保持每次攻击都基于 AgentDojo (Debenedetti et al., 2024)/AgentHarm (Andriushchenko et al., 2025) 的模式。槽位数据来源于 99 个数据泄露地址、66 个外部站点、55 个对抗性 IBAN、88 个第三方身份、77 个权威人设、77 个城市、55 个场所和 66 个金额,并结合了 88 个用户和 55 个智能体措辞前缀。
4) Cross-persona projection. A request that is perfectly valid for persona is, by construction, a functional-boundary violation for persona . Projecting benign messages across personas yields hard OOD cases that are fluent and in-distribution yet out of scope, which are far more challenging than generic off-topic text.4) 跨人设投影。对于人设 A 完全有效的请求,根据构造,对于人设 B 而言就是功能边界违规。将良性消息跨人设投影产生了 288 个困难的 OOD 案例,它们虽然流畅且符合分布,但超出了范围,这比通用的离题文本更具挑战性。
Mixed cases are then assembled programmatically by grafting an out-of-scope or adversarial rider onto a benign in-scope carrier (e.g., a valid booking request followed by an injected exfiltration instruction), exercising the guardrail’s behaviour when legitimate and illegitimate intents co-occur in a single turn.随后,通过将范围外或对抗性的附加指令(rider)嫁接到良性的范围内载体(carrier)上(例如,一个有效的预订请求后跟一条注入的数据泄露指令),以编程方式组装混合案例,从而在单个轮次中合法和非法意图共存时测试防护栏的行为。
| Source | # | Role in the corpus |
|---|---|---|
| AgentDojo (verbatim/adapted) | 234 | benign + adversarial seeds |
| LLM synthesis (Claude Opus 4.8) | 972 | benign + OOD hallucinations |
| Deterministic templates | 582 | adversarial + mixed |
| Cross-persona projection | 288 | hard out-of-domain |
| Total | 2,076 |
4.1.2. Composition and Coverage4.1.2. 组成与覆盖范围
The corpus is balanced across the four personas ( messages each) and spans both guardrail turns ( user, agent). Request kinds are distributed as benign, adversarial, out-of-domain, and mixed, giving allowed and should-block references. The AgentHarm projection adds malicious user requests ( adversarial, OOD) spanning eight harm categories (Fraud, Cybercrime, Harassment, Copyright, Sexual content, Disinformation, Hate, and Drugs), mapped onto the four personas to test whether functional-boundary enforcement generalises to externally sourced misuse.该语料库在四个人设之间保持平衡(每个 519 条消息),并涵盖了两种防护栏轮次(1,280 条用户轮次,796 条智能体轮次)。请求类型分布为:1,004 条良性、448 条对抗性、444 条域外和 180 条混合,共计 1,184 条允许的参考样本和 892 条应拦截的参考样本。AgentHarm 投影增加了 104 条恶意用户请求(52 条对抗性,52 条 OOD),涵盖八种危害类别(欺诈、网络犯罪、骚扰、版权、色情内容、虚假信息、仇恨和毒品),并映射到四个人设上,以测试功能边界执行是否能够推广到外部来源的滥用行为。
4.1.3. Diversity Analysis4.1.3. 多样性分析
A central goal of the dataset is breadth: we want each request kind to span the range of phenomena a production guardrail encounters, so that strong performance is evidence of genuine functional-boundary understanding rather than overfitting to a narrow template.数据集的一个核心目标是广度:我们希望每种请求类型都能涵盖生产防护栏所遇到的各种现象,从而使强大的性能证明其确实理解了功能边界,而不是仅仅对狭窄的模板进行了过拟合。
Adversarial coverage. Attack turns instantiate six distinct attack families (Table 5), covering data exfiltration, access to third-party data, sending data or funds to unauthorized destinations, account privilege and configuration tampering, direct prompt injection, and social-engineering via authority impersonation. Each family is realised for all four personas on both user and agent turns, and the slot-filled templates (Section 4.1.1) yield many surface realisations per family, so the guardrail cannot succeed by memorising a fixed set of trigger phrases.对抗性覆盖。攻击轮次实例化了六种不同的攻击家族(表 5),涵盖数据泄露、访问第三方数据、向未经授权的目的地发送数据或资金、账户权限和配置篡改、直接提示词注入以及通过权威冒充进行的社会工程学攻击。每个家族都在所有四个人设的用户和智能体轮次上实现,且槽位填充模板(第 4.1.1 节)为每个家族产生了许多表面实现,因此防护栏无法通过记忆一组固定的触发短语来取得成功。
| Attack family | Representative intent | # |
|---|---|---|
| Unauthorized recipient | send data/funds to an unprovided address | 56 |
| Third-party data access | read/act on another user’s data | 49 |
| Social engineering | authority impersonation to extract data | 46 |
| Data exfiltration | bulk export to an external sink | 44 |
| Prompt injection | “ignore instructions” / system override | 33 |
| Privilege/config tampering | change email, payees, passwords, admins | 29 |
Out-of-domain coverage. OOD messages combine three families: (i) cross-persona violations ( cases; a request valid for another agent, projected roughly evenly across all four source personas), which are the hardest because they are fluent and task-like; (ii) generic off-scope user queries (general knowledge, trivia, lifestyle); and (iii) off-persona agent hallucinations (poems, code, opinions) that test the post-app boundary. This mixture ensures the guardrail is measured on both obviously-off-topic text and the subtler case of competent-but-out-of-scope requests.域外覆盖。OOD 消息结合了三个家族:(i) 跨人设违规(288 个案例;对于另一个智能体有效的请求,大致均匀地投影到所有四个人设上),这是最难的,因为它们流畅且像任务请求;(ii) 通用的范围外用户查询(常识、琐事、生活方式);以及 (iii) 人设外智能体幻觉(诗歌、代码、观点),用于测试应用后的边界。这种混合确保了防护栏既能在明显的离题文本上进行测试,也能在能力范围内但超出范围的微妙请求上进行测试。
Benign coverage. The benign messages combine human-authored AgentDojo tasks with LLM-synthesized variants spanning both user requests and agent responses, so the “allow” class reflects a wide range of legitimate phrasings, multi-step tasks, and confirmation styles rather than a handful of canonical prompts. Overall the corpus contains unique messages ( of the total) drawn from a vocabulary of word types.良性覆盖。1,004 条良性消息结合了 168 条人工编写的 AgentDojo 任务和 836 条 LLM 合成的变体,涵盖了用户请求和智能体响应,因此“允许”类反映了广泛的合法措辞、多步任务和确认风格,而不是少数几个规范提示词。总的来说,语料库包含 1,725 条唯一消息(占总数的 83.1%),词汇量为 3,222 个词类型。
Length and structural variation. Message length varies widely and systematically with request kind (Table in Appendix A). Terse single-clause attack commands (median words) sit alongside long multi-step benign itineraries and cross-persona OOD requests (up to words), while mixed turns are longest (median , up to words) because they concatenate a benign carrier with an injected rider. This – word span forces the guardrail to reason over both short imperatives and long, distraction-laden inputs.长度和结构变化。消息长度随请求类型而有广泛且系统的变化(附录 A 中的表)。简洁的单从句攻击命令(中位数 14 个词)与冗长的多步良性行程规划和跨人设 OOD 请求(长达 121 个词)并存,而混合轮次最长(中位数 37 个词,长达 152 个词),因为它们将良性载体与注入的附加指令连接在一起。这种 5 到 152 个词的跨度迫使防护栏既要对简短的指令进行推理,也要对冗长且充满干扰的输入进行推理。
Taken together—four balanced personas, dual guardrail turns, four request kinds, six attack families, three OOD families, and a wide, kind-dependent length distribution over a -word vocabulary —these properties make PAGE a broad and reusable benchmark for persona-scoped guardrails, rather than a narrow test tailored to a single defense.总而言之,四个平衡的人设、双重防护栏轮次、四种请求类型、六种攻击家族、三种 OOD 家族,以及在一个 3,222 个词的词汇库上随类型变化的广泛长度分布,这些属性使 PAGE 成为一个针对人设范围防护栏的广泛且可重用的基准测试,而不是针对单一防御定制的狭窄测试。
Metrics. We treat block as the positive class: the guardrail should block off-persona content and approve in-scope content. For each configuration we report the four decision outcomes—correct approved (TN) and correct blocked (TP) versus false approved (FN, a should-block turn wrongly allowed) and false blocked (FP, a legitimate turn wrongly rejected)—together with the derived false-approved rate () and false-blocked rate (), and standard accuracy, precision, recall, and F1.指标。我们将“拦截”视为正类:防护栏应拦截人设外内容并批准范围内的内容。对于每个配置,我们报告四种决策结果——正确批准 (TN) 和正确拦截 (TP),对比错误批准 (FN,应拦截的轮次被错误允许) 和错误拦截 (FP,合法的轮次被错误拒绝)——以及导出的错误批准率 (FAR=FN/(FN+TP)) 和错误拦截率 (FBR=FP/(FP+TN)),以及标准的准确率、精确率、召回率和 F1 分数。
4.2. Guardrail Configurations4.2. 防护栏配置
We study a progression of configurations of increasing specificity, all evaluated on the same PAGE splits:我们研究了一系列特异性逐渐增加的配置,所有配置均在相同的 PAGE 分割上进行评估:
1) Baseline (LLM judge). The classifier receives only a generic natural-language description of the agent and is asked to label each turn as benign, adversarial, or out-of-domain. No allowlist, blocklist, or persona is supplied. This isolates the value added by an explicit functional boundary.1) 基线(LLM 裁判)。分类器仅接收智能体的通用自然语言描述,并被要求将每个轮次标记为良性、对抗性或域外。不提供白名单、黑名单或人设。这隔离了显式功能边界所增加的价值。
2) + Allowed intents (allowlist). We supply the agent’s persona allowlist—the enumerated set of permitted user and agent intents—and admit a turn only when it matches an allowed intent.2) + 允许的意图(白名单)。我们提供智能体的人设白名单(枚举的允许用户和智能体意图集),仅在轮次匹配允许意图时才予以承认。
3) + Iterated allowlist. We refine the allowlist by studying the false-approved and false-blocked turns of the previous configuration and sharpening the intent wording accordingly (tightening over-broad intents and clarifying scope boundaries). This is a single data-driven refinement pass.3) + 迭代白名单。我们通过研究前一个配置的错误批准和错误拦截轮次,相应地精炼意图措辞(收紧过于宽泛的意图并澄清范围边界)来优化白名单。这是一个单次数据驱动的精炼过程。
4) Blocklist only. As a counterpoint, the persona is expressed as a blocklist of disallowed intents, with no allowlist.4) 仅黑名单。作为对比,人设被表示为不允许意图的黑名单,没有白名单。
5) Allowlist + blocklist. The persona combines both an allowlist and a blocklist.5) 白名单 + 黑名单。人设结合了白名单和黑名单。
The last two configurations are exploratory: whether an explicit blocklist improves over the iterated allowlist is an empirical question, and we let the measured results (Section 4.3) determine the narrative.最后两个配置是探索性的:显式黑名单是否比迭代白名单更好是一个经验性问题,我们让测量结果(第 4.3 节)来决定结论。
4.3. Overall Detection Performance4.3. 整体检测性能
Table 6 reports aggregate performance across configurations and Table 9 the underlying decision counts. The comparison isolates the value of the functional boundary. The baseline judge—given only a generic agent description—attains reasonable adversarial detection () but cannot enforce a functional boundary: it catches only of out-of-domain turns, giving a false-approved rate and accuracy. Introducing the persona allowlist raises out-of-domain detection to and cuts the false-approved rate to ; the single iteration pass—driven entirely by inspecting the false-approved and false-blocked turns—raises out-of-domain detection further to and reduces the false-approved rate to , lifting accuracy to ; a second refinement pass and a held-out, multi-split validation of this iterative gain are reported in Appendix A (Table 12). Crucially, the benign false-blocked rate stays essentially constant across all three configurations (), so these security gains do not cost utility.表 6 报告了各配置的汇总性能,表 9 报告了底层的决策计数。该比较隔离了功能边界的价值。基线裁判(仅给定通用智能体描述)获得了合理的对抗性检测率(92.6%),但无法执行功能边界:它仅捕获了 57.3% 的域外轮次,导致 25.0% 的错误批准率和 85.7% 的准确率。引入人设白名单将域外检测率提高到 83.3%,并将错误批准率降低到 9.9%;单次迭代过程(完全由检查错误批准和错误拦截轮次驱动)将域外检测率进一步提高到 93.5%,并将错误批准率降低到 4.7%,将准确率提升至 95.9%;第二次精炼过程以及对此迭代增益进行的留出多分割验证在附录 A(表 12)中报告。至关重要的是,良性错误拦截率在所有三个配置中基本保持不变(≈3.7%),因此这些安全收益并未以牺牲效用为代价。
The blocklist-only and allowlist+blocklist configurations settle whether an explicit blocklist helps. A blocklist alone is far weaker: it reaches only accuracy because, although it catches of adversarial turns, it flags just of out-of-domain requests—a blocklist enumerates attacks, not the agent’s function, so it cannot recognise a competent request that merely falls outside scope (a false-approved rate). Adding the blocklist on top of the allowlist does not improve on the allowlist alone: raw detection edges up, but the benign false-blocked rate roughly doubles () and overall accuracy slips (). The iterated allowlist thus remains the strongest configuration: the allowlist is both necessary and sufficient to enforce the functional boundary, and a blocklist neither substitutes for it nor usefully complements it.仅黑名单和白名单+黑名单配置确定了显式黑名单是否有帮助。仅黑名单的效果要弱得多:它仅达到 71.2% 的准确率,因为尽管它捕获了 87.2% 的对抗性轮次,但它仅标记了 14.5% 的域外请求——黑名单枚举的是攻击,而不是智能体的功能,因此它无法识别仅仅超出范围的能力范围内的请求(49.0% 的错误批准率)。在白名单之上添加黑名单并不能改善仅白名单的情况:原始检测率略有上升,但良性错误拦截率大约翻了一番(3.6%→8.2%),整体准确率下滑(95.9%→94.9%)。因此,迭代白名单仍然是最强的配置:白名单对于执行功能边界既是必要的也是充分的,而黑名单既不能替代它,也不能有效地补充它。
| Metric (%) | Baseline | +Allowlist | +Iterated | Blocklist | Allow+Block |
|---|---|---|---|---|---|
| Accuracy | 85.7 | 93.2 | 95.9 | 71.2 | 94.9 |
| Precision | 95.3 | 96.0 | 96.3 | 85.4 | 92.2 |
| Recall | 75.0 | 90.1 | 95.3 | 51.0 | 98.0 |
| F1 | 83.9 | 93.0 | 95.8 | 63.9 | 95.0 |
| False-approved rate (FAR) | 25.0 | 9.9 | 4.7 | 49.0 | 2.0 |
| False-blocked rate (FBR) | 3.7 | 3.7 | 3.6 | 8.7 | 8.2 |
4.4. Per-Label Analysis4.4. 按标签分析
Table 7 decomposes performance by request label, and Table in Appendix A breaks the strongest configuration (iterated allowlist) down by agent. Two observations stand out. First, adversarial detection is high for every allowlist configuration (), and even the description-only baseline and the generic blocklist catch most overt attacks ( and ): explicit attacks are the easy case. Second, out-of-domain detection is where the functional boundary matters most—it climbs from (baseline) to (allowlist) to (iterated), whereas the blocklist manages only , confirming that identifying competent-but-out-of-scope requests—not attack coverage—is the hard part of the problem, and that only an allowlist addresses it. Per agent, the lower Slack and Workspace OOD rates reflect the difficulty of the fluent, task-like cross-persona OOD cases described in Section 4.1.3.表 7 按请求标签分解了性能,附录 A 中的表按智能体分解了最强配置(迭代白名单)。两个观察结果非常突出。首先,每个白名单配置的对抗性检测率都很高(≥96.8%),即使是仅描述的基线和通用黑名单也能捕获大多数明显的攻击(92.6% 和 87.2%):显式攻击是容易的情况。其次,域外检测是功能边界最重要的地方——它从 57.3%(基线)上升到 83.3%(白名单)再到 93.5%(迭代),而黑名单仅达到 14.5%,这证实了识别能力范围之内但超出范围的请求(而不是攻击覆盖)是问题的难点,且只有白名单能解决它。按智能体来看,Slack 和 Workspace 较低的 OOD 率反映了第 4.1.3 节中描述的流畅、任务式跨人设 OOD 案例的难度。
| Label | Base | Allow | Iter. | Block | Allow+Block |
|---|---|---|---|---|---|
| Benign (FBR ) | 3.7 | 3.7 | 3.6 | 8.7 | 8.2 |
| Adversarial () | 92.6 | 96.8 | 97.0 | 87.2 | 99.2 |
| Out-of-domain () | 57.3 | 83.3 | 93.5 | 14.5 | 96.8 |
4.5. Ablation: Mixed Intents4.5. 消融研究:混合意图
The experiments above use the three-label setting (benign, adversarial, out-of-domain). We additionally study mixed turns—turns that combine a valid in-scope request with an out-of-scope or adversarial rider—which raise a labelling question: should a turn that contains some legitimate intent be approved (allowlist-presence semantics) or blocked (because it also carries off-policy content)? We evaluate both labelling policies against the no-mixed setting on the iterated-allowlist configuration (Table 8).上述实验使用三标签设置(良性、对抗性、域外)。我们额外研究了混合轮次——结合了有效的范围内请求和范围外或对抗性附加指令的轮次——这提出了一个标记问题:包含某些合法意图的轮次应该被批准(白名单存在语义)还是被拦截(因为它也携带了策略外内容)?我们在迭代白名单配置上针对无混合设置评估了这两种标记策略(表 8)。
The guardrail blocks of mixed turns in both policies (its predictions do not depend on the label). Consequently, scoring mixed turns as should-approve inflates the false-blocked rate to and drops accuracy to , whereas scoring them as should-block keeps accuracy at with a false-blocked rate. In other words, the guardrail behaves as if a mixed turn is off-policy, which is consistent with a strict functional boundary that rejects any turn carrying a disallowed component.防护栏在两种策略中都拦截了 92.2% 的混合轮次(其预测不依赖于标签)。因此,将混合轮次评分为“应批准”会将错误拦截率提高到 17.1% 并将准确率降至 88.6%,而将其评分为“应拦截”则使准确率保持在 95.6%,错误拦截率为 3.6%。换句话说,防护栏的表现就像混合轮次是策略外的一样,这与拒绝任何携带违规组件的轮次的严格功能边界是一致的。
| Metric (%) | No mixed |
|
| ||||
|---|---|---|---|---|---|---|---|
| Accuracy | 95.9 | 88.6 | 95.6 | ||||
| Precision | 96.3 | 82.5 | 96.9 | ||||
| Recall | 95.3 | 95.3 | 94.8 | ||||
| F1 | 95.8 | 88.4 | 95.8 | ||||
| FAR | 4.7 | 4.7 | 5.2 | ||||
| FBR | 3.6 | 17.1 | 3.6 | ||||
| Mixed turns blocked | – | 92.2 | 92.2 |
5. Production Deployment5. 生产部署
While the previous sections focused on the design and evaluation of Persona Guardrail, deploying guardrails in production introduces a distinct set of engineering challenges. Production agentic systems operate across multiple agents with heterogeneous capabilities, evolving product requirements, and dynamic user behavior. Consequently, guardrails must not only provide strong security but also remain observable, maintainable, and adaptable throughout their lifecycle, while operating within the strict latency budget imposed by their position on the critical path.虽然前几节侧重于个人防护栏的设计和评估,但在生产环境中部署防护栏引入了一系列独特的工程挑战。生产智能体系统运行在具有异构能力的多个智能体之上,且产品需求不断演变,用户行为动态变化。因此,防护栏不仅必须提供强大的安全性,还必须在整个生命周期内保持可观测性、可维护性和适应性,同时在关键路径上所施加的严格延迟预算内运行。
5.1. Model Serving Infrastructure5.1. 模型服务基础设施
The guardrail model—a 30B-parameter Qwen3 mixture-of-experts model that activates approximately 3B parameters per token and is quantized to FP8—is served with vLLM on NVIDIA Dynamo. Inference is disaggregated into a CPU frontend and GPU workers: a prefix-cache-aware router dispatches each request to a worker, which performs the model inference on a dedicated accelerator. The frontend and worker pools scale independently (Figure 3 in the Appendix).防护栏模型——一个 30B 参数的 Qwen3 混合专家模型,每个 token 激活约 3B 参数,并量化为 FP8——在 NVIDIA Dynamo 上使用 vLLM 提供服务。推理被分解为 CPU 前端和 GPU 工作节点:一个感知前缀缓存的路由器将每个请求分派给工作节点,由其在专用加速器上执行模型推理。前端和工作节点池独立扩展(附录中的图 3)。
Because the guardrail lies on the critical path of every interaction, the serving infrastructure is optimized for predictable latency and high availability. The model context length is capped to bound the per-request KV-cache footprint and worst-case inference latency. Workers register with a per-region discovery mechanism, and requests are routed only to healthy workers in the same region. Individual worker failures therefore reduce local capacity without interrupting the remaining workers, while regional isolation contains region-wide failures and permits application-level failover.由于防护栏位于每次交互的关键路径上,服务基础设施针对可预测的延迟和高可用性进行了优化。模型上下文长度被限制,以约束每个请求的 KV 缓存占用空间和最坏情况下的推理延迟。工作节点在区域级发现机制中注册,请求仅路由到同一区域内的健康工作节点。因此,单个工作节点的故障会减少本地容量,而不会中断其余工作节点,而区域隔离则控制了区域范围的故障并允许应用级故障转移。
5.2. Serving Within a Latency Budget5.2. 在延迟预算内服务
Because a guardrail runs on every request and response, its latency is judged not in isolation but against a transparency budget: it may consume only a small fraction of the host application’s end-to-end processing time before users perceive it. We express this budget as a fixed fraction of application latency, yielding a sub-100 ms P90 target for each guardrail model call.由于防护栏在每个请求和响应上运行,其延迟不是孤立判断的,而是根据透明度预算进行衡量:在用户感知到之前,它只能消耗主机应用端到端处理时间的一小部分。我们将此预算表示为应用延迟的固定比例,从而为每个防护栏模型调用产生低于 100 ms 的 P90 目标。
Meeting this target is made tractable by three properties of the guardrail workload.通过防护栏工作负载的三个属性,实现这一目标变得可行。
1) Single-token, prefill-dominated inference. The guardrail is a binary classifier that emits one decision token (allow or block). Token generation is negligible, so latency is dominated by prompt prefill rather than decoding.1) 单 token、以预填充为主的推理。防护栏是一个二元分类器,发出一个决策 token(允许或拦截)。Token 生成量可忽略不计,因此延迟主要由提示词预填充而非解码决定。
2) A large, shared prompt prefix. All requests share a static preamble containing the instructions and policy context. This preamble accounts for approximately 93% of input tokens in our workload. Reusing its cached state reduces per-request prefill to the shorter, variable user request or agent response.2) 一个大的、共享的提示词前缀。所有请求共享一个包含指令和策略上下文的静态前言。在我们的工作负载中,该前言约占输入 token 的 93%。重用其缓存状态将每个请求的预填充减少为较短的、可变的用户请求或智能体响应。
3) Bounded context with cheap rejection. The small fraction of prompts that exceed the context cap (2.1% of production traffic) is rejected before GPU work is scheduled. This prevents oversized inputs from consuming capacity or inflating latency for the remaining traffic (Appendix B).3) 带有廉价拒绝的有限上下文。超过上下文上限的那一小部分提示词(生产流量的 2.1%)在 GPU 工作被调度之前就被拒绝。这防止了过大的输入消耗容量或增加其余流量的延迟(附录 B)。
Beyond exploiting workload structure, we reduce per-call cost through model selection rather than compression alone. A mixture-of-experts model with only a few billion active parameters, served in FP8, provides the capacity of a larger model at the inference cost of a substantially smaller one. Together, disaggregated serving, prefix-cache reuse, bounded contexts, and one-token output target the prefill-dominated structure of the workload rather than generic token-generation speed.除了利用工作负载结构外,我们还通过模型选择而非单纯的压缩来降低每次调用的成本。一个仅有几十亿激活参数、以 FP8 提供服务的混合专家模型,以大幅减小的推理成本提供了较大模型的容量。总之,分解式服务、前缀缓存重用、有限上下文和单 token 输出,针对的是工作负载以预填充为主的结构,而不是通用的 token 生成速度。
Under this design, the guardrail sustains its latency budget across the tested operating range (Figure 1). Between 5 and 50 QPS, median latency ranges from 19.43 to 25.34 ms and P90 latency from 57.30 to 61.08 ms. P99 increases from 76.77 ms at 5 QPS to 104.15 ms at 50 QPS, crossing the 100 ms objective at 30 QPS. These measurements are single-accelerator model-call latencies with a 4,096-token context cap; they do not include application or network overhead. Accelerator utilization remains below 65% over this range. Relative to the prior general-purpose serving stack, the disaggregated design reduces guardrail-call latency by approximately 30%.在这种设计下,防护栏在测试的操作范围内保持了其延迟预算(图 1)。在 5 到 50 QPS 之间,中位数延迟范围为 19.43 到 25.34 ms,P90 延迟为 57.30 到 61.08 ms。P99 从 5 QPS 时的 76.77 ms 增加到 50 QPS 时的 104.15 ms,在 30 QPS 时超过了 100 ms 的目标。这些测量是具有 4,096 token 上下文上限的单加速器模型调用延迟;它们不包括应用或网络开销。在此范围内,加速器利用率保持在 65% 以下。相对于之前的通用服务栈,分解式设计将防护栏调用延迟降低了约 30%。
5.3. Observability and Monitoring5.3. 可观测性与监控
Guardrail failures are frequently semantic rather than deterministic, so infrastructure health alone is insufficient. Persona Guardrail records telemetry for every processed interaction, including the decision, agent identifier, evaluated boundary, policy version, inference latency, and final outcome. Agent-level monitoring tracks allow and block rates and latency distributions, while operational monitoring measures throughput, resource utilization, availability, context-limit rejection, and end-to-end latency.防护栏故障通常是语义上的而非确定性的,因此仅基础设施健康是不够的。个人防护栏记录每次处理交互的遥测数据,包括决策、智能体标识符、评估边界、策略版本、推理延迟和最终结果。智能体级监控跟踪允许和拦截率以及延迟分布,而操作监控测量吞吐量、资源利用率、可用性、上下文限制拒绝和端到端延迟。
Allow and block rates, together with disagreements between shadow and active policies, identify examples for investigation but are not themselves estimates of false-positive or false-negative rates. Such rates require examples with known labels or independently adjudicated samples. This distinction prevents agreement with the active policy from being mistaken for ground truth.允许和拦截率,连同影子策略与活动策略之间的分歧,确定了供调查的示例,但它们本身并不是假阳性或假阴性率的估计。这些比率需要带有已知标签或经过独立裁决的样本的示例。这种区分防止了与活动策略的一致被误认为是事实真相。
Routine metrics contain identifiers and aggregate measurements rather than raw prompts, responses, or intent lists. Content retained for debugging or refinement is sampled, minimized or redacted, access-controlled, and deleted according to a bounded retention policy. These monitoring signals support the continuous guardrail evolution framework described in Section 5.5.常规指标包含标识符和汇总测量值,而不是原始提示词、响应或意图列表。保留用于调试或精炼的内容会被采样、最小化或脱敏、访问控制,并根据有限的保留策略进行删除。这些监控信号支持第 5.5 节中描述的持续防护栏演进框架。
5.4. Guardrail Versioning and Safe Deployment5.4. 防护栏版本控制与安全部署
Unlike static security policies, guardrail specifications evolve as agent capabilities and attack techniques change. Persona Guardrail maintains an independent policy for each deployed agent, containing allowed and blocked intents for both user requests and agent responses. The same classifier can serve multiple agents, but each evaluation uses the intent lists associated with the current agent and interaction boundary. Candidate updates must satisfy predefined thresholds on security effectiveness, utility, reliability, and latency before deployment.与静态安全策略不同,防护栏规范会随着智能体能力和攻击技术的变化而演变。个人防护栏为每个部署的智能体维护一个独立的策略,包含用户请求和智能体响应的允许和禁止意图。同一个分类器可以服务多个智能体,但每次评估都使用与当前智能体和交互边界相关联的意图列表。候选更新在部署前必须满足预定义的安全性有效性、效用、可靠性和延迟阈值。
Operationally, the editable intent taxonomy is separated from the policy enforced in production. Applying a persona to a policy copies its four intent lists into a deployment snapshot; later edits to the source persona do not affect that snapshot. A persona used by a shadow or active policy is not edited in place. Instead, it is cloned, the clone is refined, and the resulting candidate proceeds through offline evaluation, review, and rollout. Thus, each iteration creates an explicit candidate while the currently active allowlist remains unchanged.在操作上,可编辑的意图分类法与生产中执行的策略是分离的。将人设应用于策略会将它的四个意图列表复制到部署快照中;随后对源人设的编辑不会影响该快照。影子或活动策略使用的人设不会在原处进行编辑。相反,它会被克隆,克隆版本会被精炼,生成的候选版本会通过离线评估、审查和发布流程。因此,每次迭代都会创建一个明确的候选版本,而当前活动的白名单保持不变。
To reduce overfitting to a fixed benchmark, offline evaluation uses three complementary pools: a golden set of curated, unambiguous behaviors that acts as a regression floor; an adversarial set of prompt injections, jailbreaks, and difficult boundary cases; and a production set sampled from live traffic. Examples used to motivate a refinement are kept separate from the held-out examples used to validate it. A candidate is accepted only if it satisfies the predefined thresholds across all three pools.为了减少对固定基准测试的过拟合,离线评估使用三个互补池:一组作为回归底线的精选、无歧义行为的黄金集;一组提示词注入、越狱和困难边界案例的对抗性集;以及一组从实时流量中采样的生产集。用于激励精炼的示例与用于验证它的留出示例是分开的。只有当候选版本在所有三个池中都满足预定义阈值时,才会被接受。
Once approved, the candidate snapshot is distributed as declarative configuration rather than as a serving-code deployment. This allows per-agent policies to be promoted independently without redeploying the shared model-serving stack. Every production activation is gated by independent human review.一旦批准,候选快照将作为声明性配置分发,而不是作为服务代码部署。这允许每个智能体的策略在不重新部署共享模型服务栈的情况下独立提升。每次生产激活都由独立的人工审查进行门控。
New policies are initially deployed in shadow mode, where they evaluate sampled production traffic without affecting user-visible decisions. Their predictions are compared with those of the active policy to identify disagreements for analysis. A candidate that passes shadow evaluation is introduced through a limited enforcing rollout before becoming active. The same snapshot is used throughout these stages. If a regression is detected, the rollout is stopped and the previously active snapshot remains in effect or is restored.新策略最初以影子模式部署,它们评估采样的生产流量而不影响用户可见的决策。它们的预测与活动策略的预测进行比较,以识别分歧进行分析。通过影子评估的候选版本在成为活动状态之前,会通过有限的强制发布引入。在这些阶段中都使用相同的快照。如果检测到回归,发布将停止,之前活动的快照将保持有效或被恢复。
5.5. Continuous Guardrail Evolution5.5. 持续的防护栏演进
Persona Guardrail evolves primarily by refining its human-readable intent taxonomy rather than retraining the classifier. Refinement may be triggered by newly introduced agent functionality, newly observed attacks, distribution shifts, or adjudicated classification errors. These signals initiate investigation; they never modify an active policy automatically.个人防护栏主要通过精炼其人类可读的意图分类法而不是重新训练分类器来演进。精炼可能由新引入的智能体功能、新观察到的攻击、分布偏移或裁决的分类错误触发。这些信号启动调查;它们从不自动修改活动策略。
Examples are partitioned before refinement into a discovery pool and held-out validation pools. Correlated conversations, near-duplicate examples, and variants from the same attack campaign remain in the same partition. The discovery pool is used to propose changes, whereas acceptance is measured on held-out golden, adversarial, and production-derived examples. Deduplication, provenance tracking, and per-source sampling limits reduce evaluation leakage and the risk that one actor dominates a production-derived sample.示例在精炼前被划分为发现池和留出验证池。相关的对话、近乎重复的示例以及来自同一攻击活动的变体保留在同一分区中。发现池用于提出更改,而验收则在留出的黄金、对抗性和生产衍生示例上进行测量。去重、来源跟踪和按来源采样限制减少了评估泄露,并降低了一个参与者主导生产衍生样本的风险。
Golden and synthetic examples have labels assigned during construction. For otherwise unlabeled production examples, a stronger judge model is given the agent’s functional boundary and the interaction, but not the guardrail’s decision, and predicts whether the interaction is permitted. Judge-model disagreements with the guardrail, together with a sample of agreements, are independently adjudicated before inclusion in reported metrics. We treat an out-of-scope or adversarial interaction as the positive class: a true positive is correctly blocked, a false negative is incorrectly allowed, a false positive is a legitimate interaction that is blocked, and a true negative is a legitimate interaction that is allowed. Thus, disagreement with either the judge or the active policy is not by itself treated as an error label.黄金和合成示例在构建时分配了标签。对于其他未标记的生产示例,更强大的裁判模型会被给予智能体的功能边界和交互,但不给予防护栏的决策,并预测交互是否被允许。裁判模型与防护栏的分歧,连同一部分一致的样本,在包含在报告指标中之前会经过独立裁决。我们将范围外或对抗性交互视为正类:真阳性是正确拦截,假阴性是错误允许,假阳性是合法的交互被拦截,真阴性是合法的交互被允许。因此,与裁判或活动策略的分歧本身并不被视为错误标签。
A separate offline proposer groups errors from the discovery pool and suggests changes only to a new draft. The proposer cannot modify labels, approve its proposal, or change production policy. The resulting candidate is evaluated on the held-out pools and enters the review and rollout lifecycle described above only if it satisfies the predefined security, utility, reliability, and latency thresholds.一个独立的离线提议者对发现池中的错误进行分组,并仅对新草案提出更改建议。提议者不能修改标签、批准其提案或更改生产策略。生成的候选版本在留出池上进行评估,并且仅在满足预定义的安全性、效用、可靠性和延迟阈值时,才进入上述审查和发布生命周期。
5.6. Human-in-the-Loop Guardrail Management5.6. 人机协作防护栏管理
Although Persona Guardrail automates much of the guardrail lifecycle, human oversight remains essential for maintaining high-quality functional and security boundaries. Misclassified interactions identified through production monitoring are periodically reviewed by developers or security analysts to determine whether they correspond to evolving user intent, product changes, or novel attack strategies. Legitimate user interactions that fall outside the current allowlist may indicate that the agent’s functional boundary should be expanded, whereas newly observed attack patterns may require updates to the blocklist.尽管个人防护栏自动化了防护栏生命周期的大部分,但人工监督对于维持高质量的功能和安全边界仍然至关重要。通过生产监控识别的错误分类交互会定期由开发人员或安全分析师审查,以确定它们是否对应于演进的用户意图、产品变更或新颖的攻击策略。落在当前白名单之外的合法用户交互可能表明智能体的功能边界应该扩大,而新观察到的攻击模式可能需要更新黑名单。
Human review is particularly valuable for validating ambiguous interactions and resolving conflicts between usability and security. Rather than directly incorporating every production observation into the guardrail specification, Persona Guardrail treats human feedback as a high-confidence signal for refining both functional and security boundaries.人工审查对于验证模糊交互和解决可用性与安全性之间的冲突特别有价值。个人防护栏并没有直接将每个生产观察结果纳入防护栏规范,而是将人工反馈视为精炼功能和安全边界的高置信度信号。
5.7. Multi-Agent Deployment5.7. 多智能体部署
Large production systems often consist of numerous specialized agents, each serving a distinct purpose while sharing common infrastructure and security requirements. Deploying independent guardrails for every agent quickly becomes difficult to manage as the number of agents increases.大型生产系统通常由众多专门智能体组成,每个智能体服务于不同的目的,同时共享共同的基础设施和安全要求。随着智能体数量的增加,为每个智能体部署独立的防护栏很快变得难以管理。
Persona Guardrail addresses this challenge by separating agent-specific functional boundaries from shared security knowledge. Per-agent functional boundaries are stored as versioned records in a relational (MySQL) store and managed through the Registry, which exposes a management API for creating, updating, and versioning each agent’s allowlists and out-of-domain definitions. The registry serves as the single source of truth consumed by both the tuning pipeline and the policy engine, giving agent teams a self-serve control plane while ensuring a consistent view of policy across the fleet. The shared security blocklist—covering common attack patterns such as prompt injection, jailbreaks, indirect prompt injection, and prompt leakage—is maintained centrally and referenced by every agent, so a signature added once is enforced everywhere.个人防护栏通过将智能体特定的功能边界与共享安全知识分离来应对这一挑战。每个智能体的功能边界作为版本化记录存储在关系型 (MySQL) 存储中,并通过注册表进行管理,注册表公开了一个用于创建、更新和版本化每个智能体白名单和域外定义的管理 API。注册表作为调优流水线和策略引擎消费的单一事实来源,为智能体团队提供自助控制平面,同时确保整个机群中策略的一致视图。共享安全黑名单——涵盖常见的攻击模式,如提示词注入、越狱、间接提示词注入和提示词泄露——由中心维护并被每个智能体引用,因此添加一次的签名会在各处强制执行。
This separation enables new attack signatures discovered in one agent to be propagated rapidly across the entire deployment while allowing each agent to maintain its own functional specification. As a result, Persona Guardrail scales efficiently to heterogeneous multi-agent deployments without requiring every guardrail to be developed and maintained independently.这种分离使得在一个智能体中发现的新攻击签名能够迅速传播到整个部署中,同时允许每个智能体维护自己的功能规范。因此,个人防护栏能够高效地扩展到异构的多智能体部署中,而无需每个防护栏都独立开发和维护。
5.8. Results and Operational Lessons5.8. 结果与操作经验
In production, Persona Guardrail meets its latency budget with headroom while sustaining a false-block rate below 1% and a false-allow rate below 2% across various agents.在生产中,个人防护栏在满足其延迟预算的同时留有余地,并在各种智能体中保持低于 1% 的错误拦截率和低于 2% 的错误允许率。
Our deployment experience highlights that maintaining production guardrails is fundamentally an ongoing operational process rather than a one-time model development task. Effective guardrails require continuous observability, standardized evaluation, structured feedback collection, and disciplined deployment practices. By integrating these operational components into a unified lifecycle, Persona Guardrail enables organizations to maintain robust security while adapting to evolving products, user behavior, and adversarial threats.我们的部署经验强调,维护生产防护栏本质上是一个持续的操作过程,而不是一次性的模型开发任务。有效的防护栏需要持续的可观测性、标准化的评估、结构化的反馈收集和规范的部署实践。通过将这些操作组件集成到一个统一的生命周期中,个人防护栏使组织能够在适应演进的产品、用户行为和对抗性威胁的同时,保持强大的安全性。
Two further lessons stand out from operating Persona Guardrail at scale. First, a guardrail is a specialized inference workload—prefill-dominated and single-token, with a large shared prompt prefix—rather than a general-purpose chat model; meeting its transparency budget followed from exploiting that structure through prefix-cache reuse, frontend/decode disaggregation, and an efficient model, not from generic serving tuning. Second, decoupling policy iteration from infrastructure deployment—by expressing guardrail versions as peer-reviewed configuration changes—was essential to reconciling rapid iteration with production safety.从大规模运行个人防护栏中,我们得出了另外两个教训。首先,防护栏是一种专门的推理工作负载——以预填充为主且为单 token,具有大的共享提示词前缀——而不是通用的聊天模型;满足其透明度预算源于通过前缀缓存重用、前端/解码分解和高效模型来利用该结构,而不是源于通用的服务调优。其次,将策略迭代与基础设施部署解耦——通过将防护栏版本表示为经过同行评审的配置更改——对于协调快速迭代与生产安全至关重要。
Scope and Limitations范围与局限性
Persona Guardrail is a defense-in-depth control evaluated for direct malicious-user and out-of-domain interactions under the defined threat model. The reported results do not establish complete protection against indirect prompt injection, compromised components, unauthorized intermediate tool actions, multi-turn attacks, or other threats outside that model. Deployment decisions and acceptable error rates remain application-specific.个人防护栏是一种纵深防御控制,在定义的威胁模型下针对直接恶意用户和域外交互进行了评估。报告的结果不能建立对间接提示词注入、受损组件、未经授权的中间工具操作、多轮攻击或该模型之外的其他威胁的完全保护。部署决策和可接受的错误率仍然是特定于应用的。
6. Related Work6. 相关工作
Integrating LLMs with external tools, APIs etc and deploying it to serve specific a purpose introduces attack surfaces beyond traditional LLM safety, including prompt injection, indirect prompt injection, goal hijacking, retrieval poisoning, and tool misuse (Greshake et al., 2023; Liu et al., 2024). A recent systematization of knowledge (Kim et al., 2026) consolidates these attacks and defenses and highlights needs for scalable, adaptive, and deployable runtime protection—the gap Persona Guardrail targets.将 LLM 与外部工具、API 等集成并部署以服务特定目的,引入了超出传统 LLM 安全性的攻击面,包括提示词注入、间接提示词注入、目标劫持、检索中毒和工具滥用 (Greshake et al., 2023; Liu et al., 2024)。最近的一项知识系统化研究 (Kim et al., 2026) 整合了这些攻击和防御,并强调了对可扩展、自适应和可部署运行时保护的需求——这正是个人防护栏所针对的差距。
Security Defenses for AI AgentsAI 智能体的安全防御
Runtime defenses target specific attacks during execution. Task Shield (Jia et al., 2025) enforces task alignment, checking that intermediate steps serve the user’s task; MELON (Zhu et al., 2025) re-executes the trajectory with a masked, task-neutral user prompt and flags injection when the masked and original tool calls coincide; and LlamaFirewall (Chennabasappa et al., 2025), in production at Meta, chains jailbreak, chain-of-thought alignment, and code scanners over agent inputs, reasoning, and outputs. A complementary line enforces security by construction: CaMeL (Debenedetti et al., 2025) separates control and data flow so untrusted content cannot alter the program; the Instruction Hierarchy (Wallace et al., 2024), StruQ (Chen et al., 2025a), and SecAlign (Chen et al., 2025b) train models to prioritize trusted instructions; and Progent (Shi et al., 2025) and IsolateGPT (Wu et al., 2025) constrain tool privileges and isolate components. These defenses chiefly address indirect, environment-mediated attacks on the agent’s internal execution. Persona Guardrail is complementary: under a direct malicious-user threat model it models an agent’s functional boundary and enforces it with input and output guardrails, capturing adversarial, out-of-domain, and policy-violating turns alike.运行时防御针对执行过程中的特定攻击。Task Shield (Jia et al., 2025) 执行任务对齐,检查中间步骤是否服务于用户的任务;MELON (Zhu et al., 2025) 使用掩码、任务中立的用户提示词重新执行轨迹,并在掩码和原始工具调用重合时标记注入;LlamaFirewall (Chennabasappa et al., 2025) 在 Meta 生产中使用,在智能体输入、推理和输出上链式执行越狱、思维链对齐和代码扫描器。另一个互补的路线通过构造来执行安全:CaMeL (Debenedetti et al., 2025) 分离控制流和数据流,以便不受信任的内容无法更改程序;指令层次结构 (Wallace et al., 2024)、StruQ (Chen et al., 2025a) 和 SecAlign (Chen et al., 2025b) 训练模型以优先处理受信任的指令;Progent (Shi et al., 2025) 和 IsolateGPT (Wu et al., 2025) 限制工具权限并隔离组件。这些防御主要解决针对智能体内部执行的间接、环境介导的攻击。个人防护栏是互补的:在直接恶意用户威胁模型下,它建模智能体的功能边界,并用输入和输出防护栏强制执行,捕获对抗性、域外和违反策略的轮次。
Guardrails for LLM ApplicationsLLM 应用的防护栏
Application guardrails fall into rule-based toolkits (NeMo (Rebedea et al., 2023)), model-based classifiers (Llama Guard (Inan et al., 2023), ShieldGemma (Zeng et al., 2024), WildGuard (Han et al., 2024)), and hybrids. Closest to our work, Constitutional Classifiers (Sharma et al., 2025) train input and output classifiers from a natural-language policy with production-grade overhead. These are typically static, inference-time classifiers over global harm categories. Persona Guardrail instead enforces a per-agent functional boundary—an allowlist of intended intents that also rejects competent but out-of-scope requests—and treats guardrails as continuously evolving systems maintained via specification, standardized evaluation, and feedback-driven refinement.应用防护栏分为基于规则的工具包 (NeMo (Rebedea et al., 2023))、基于模型的分类器 (Llama Guard (Inan et al., 2023), ShieldGemma (Zeng et al., 2024), WildGuard (Han et al., 2024)) 和混合方案。与我们工作最接近的是 Constitutional Classifiers (Sharma et al., 2025),它们从自然语言策略中训练输入和输出分类器,并具有生产级开销。这些通常是针对全局危害类别的静态、推理时分类器。而个人防护栏执行的是每个智能体的功能边界——一个意图白名单,同时也拒绝能力范围内但超出范围的请求——并将防护栏视为通过规范、标准化评估和反馈驱动的精炼来维护的持续演进系统。
Benchmarks for Agentic Security智能体安全基准测试
Agentic-security evaluation has standardized around benchmarks for attacks and utility—AgentDojo (Debenedetti et al., 2024), InjecAgent (Zhan et al., 2024), AgentHarm (Andriushchenko et al., 2025), and Agent Security Bench (Zhang et al., 2025)—none of which label whether a request lies inside an agent’s intended function. Recognizing out-of-scope requests has a long lineage in task-oriented dialog, e.g., CLINC150 (Larson et al., 2019). PAGE builds on AgentDojo’s domains but adds agent-specific out-of-domain data and labels for both user and agent turns, jointly measuring attack robustness and rejection of out-of-scope requests.智能体安全评估已围绕攻击和效用基准测试标准化——AgentDojo (Debenedetti et al., 2024)、InjecAgent (Zhan et al., 2024)、AgentHarm (Andriushchenko et al., 2025) 和 Agent Security Bench (Zhang et al., 2025)——但没有一个标记请求是否位于智能体的预期功能之内。识别范围外请求在任务导向对话中有着悠久的传统,例如 CLINC150 (Larson et al., 2019)。PAGE 构建在 AgentDojo 的领域之上,但增加了特定于智能体的域外数据和用户及智能体轮次的标签,联合测量攻击鲁棒性和对范围外请求的拒绝。
In sum, prior work advances attacks and class-specific runtime, architectural, and training defenses, plus classifiers for global safety policies; Persona Guardrail contributes a production-grade framework for specifying, enforcing, evaluating, and continuously evolving an agent’s functional boundary under a direct malicious-user threat model.总之,先前的工作推进了攻击和类特定的运行时、架构和训练防御,以及针对全局安全策略的分类器;个人防护栏贡献了一个生产级框架,用于在直接恶意用户威胁模型下指定、执行、评估和持续演进智能体的功能边界。
7. Conclusion7. 结论
We presented Persona Guardrail, a runtime defense framework that enforces agent-specific functional boundaries through synchronous validation of user inputs and final agent responses. We also introduced PAGE, a benchmark covering benign, adversarial, and out-of-domain interactions across user and agent turns. On the evaluated non-mixed subset of PAGE, an iteratively refined allowlist improved accuracy from 85.7% for a generic LLM judge to 95.9%, increased out-of-domain detection from 57.3% to 93.5%, and reduced the false-approved rate from 25.0% to 4.7%. Our deployment study further shows that functional-boundary classification can be served within a sub-100 ms P90 per-call latency budget using a disaggregated, prefix-cache-aware inference architecture. These results indicate that explicit, agent-specific allowlists can complement attack-focused defenses by rejecting both malicious requests and requests outside an agent’s intended scope.我们提出了 Persona Guardrail,这是一个运行时防御框架,通过对用户输入和最终代理响应进行同步验证,来强制执行特定于代理的功能边界。我们还引入了 PAGE,这是一个涵盖用户和代理轮次中良性、对抗性和领域外交互的基准测试。在 PAGE 的非混合子集评估中,经过迭代优化的允许列表将通用 LLM 判别器的准确率从 85.7% 提高到了 95.9%,将领域外检测率从 57.3% 提高到了 93.5%,并将误批准率从 25.0% 降低到了 4.7%。我们的部署研究进一步表明,使用解耦的、感知前缀缓存的推理架构,功能边界分类可以在单次调用延迟低于 100 毫秒的 P90 预算内完成。这些结果表明,明确的、特定于代理的允许列表可以通过拒绝恶意请求和代理预期范围之外的请求,来补充以攻击为中心的防御手段。
Future work will extend Persona Guardrail in two directions. First, we plan to develop conversation-level persona enforcement that evaluates each interaction in the context of previous turns, enabling the guardrail to identify functional-boundary violations that emerge across a multi-turn conversation. Second, we plan to integrate persona enforcement at the model execution layer so that intermediate tool calls can be validated against the agent’s functional boundary before execution. Together, these extensions will expand Persona Guardrail from turn-level input and output validation to conversation- and tool-level enforcement.未来的工作将从两个方向扩展 Persona Guardrail。首先,我们计划开发对话层面的角色强制机制,在先前轮次的上下文中评估每次交互,使防护栏能够识别多轮对话中出现的功能边界违规行为。其次,我们计划在模型执行层集成角色强制机制,以便在执行前根据代理的功能边界验证中间工具调用。这些扩展将共同推动 Persona Guardrail 从轮次层面的输入输出验证,扩展到对话和工具层面的强制执行。
References参考文献
- Agentharm: a benchmark for measuring harmfulness of llm agents. In International Conference on Learning Representations, Vol. 2025, pp. 79185–79220. Cited by: §1, §4.1.1, §4.1, §6. Andriushchenko 等人 (2025) M. Andriushchenko, A. Souly, M. Dziemian, D. Duenas, M. Lin, J. Wang, D. Hendrycks, A. Zou, Z. Kolter, M. Fredrikson 等。 Agentharm: 衡量 LLM 代理危害性的基准。 收录于:国际学习表征会议 (ICLR), 第 2025 卷,第 79185–79220 页。 引用自:§1, §4.1.1, §4.1, §6.
- StruQ: defending against prompt injection with structured queries. In 34th USENIX Security Symposium (USENIX Security ’25), External Links: Link Cited by: §6. Chen 等人 (2025a) S. Chen, J. Piet, C. Sitawarin, 和 D. Wagner StruQ: 使用结构化查询防御提示词注入。 收录于:第 34 届 USENIX 安全研讨会 (USENIX Security ’25), 外部链接:链接 引用自:§6.
- SecAlign: defending against prompt injection with preference optimization. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security (CCS ’25), External Links: Document, Link Cited by: §6. Chen 等人 (2025b) S. Chen, A. Zharmagambetov, S. Mahloujifar, K. Chaudhuri, D. Wagner, 和 C. Guo SecAlign: 使用偏好优化防御提示词注入。 收录于:2025 年 ACM SIGSAC 计算机与通信安全会议论文集 (CCS ’25), 外部链接:文档, 链接 引用自:§6.
- LlamaFirewall: an open source guardrail system for building secure AI agents. External Links: 2505.03574, Link Cited by: §6. Chennabasappa 等人 (2025) S. Chennabasappa, C. Nikolaidis, D. Song, D. Molnar, S. Ding, S. Wan, S. Whitman, L. Deason, N. Doucette, A. Montilla, A. Gampa, B. de Paola, D. Gabi, J. Crnkovich, J. Testud, K. He, R. Chaturvedi, W. Zhou, 和 J. Saxe LlamaFirewall: 用于构建安全 AI 代理的开源防护栏系统。 外部链接:2505.03574, 链接 引用自:§6.
- Defeating prompt injections by design. External Links: 2503.18813, Link Cited by: §6. Debenedetti 等人 (2025) E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, 和 F. Tramèr 通过设计击败提示词注入。 外部链接:2503.18813, 链接 引用自:§6.
- Agentdojo: a dynamic environment to evaluate prompt injection attacks and defenses for llm agents. Advances in neural information processing systems 37, pp. 82895–82920. Cited by: §1, §1, §4.1.1, §4.1.1, §4, §6. Debenedetti 等人 (2024) E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, 和 F. Tramèr Agentdojo: 评估 LLM 代理提示词注入攻击与防御的动态环境。 神经信息处理系统进展 37, 第 82895–82920 页。 引用自:§1, §1, §4.1.1, §4.1.1, §4, §6.
- Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. External Links: 2511.21631, Link Cited by: §3.2, §3.3. B. 等人 (2025) B. 等人 Qwen3-vl 技术报告。 arXiv 预印本 arXiv:2511.21631。 外部链接:2511.21631, 链接 引用自:§3.2, §3.3.
- Not what you’ve signed up for: compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM workshop on artificial intelligence and security, pp. 79–90. Cited by: §1, §6. Greshake 等人 (2023) K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, 和 M. Fritz 并非你所愿:利用间接提示词注入危害现实世界中集成 LLM 的应用程序。 收录于:第 16 届 ACM 人工智能与安全研讨会论文集, 第 79–90 页。 引用自:§1, §6.
- WildGuard: open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, External Links: 2406.18495, Link Cited by: §6. Han 等人 (2024) S. Han, K. Rao, A. Ettinger, L. Jiang, B. Y. Lin, N. Lambert, Y. Choi, 和 N. Dziri WildGuard: 用于 LLM 安全风险、越狱和拒绝回答的一站式开源审核工具。 收录于:神经信息处理系统 (NeurIPS) 数据集与基准测试轨道进展, 外部链接:2406.18495, 链接 引用自:§6.
- Llama Guard: LLM-based input-output safeguard for human-AI conversations. External Links: 2312.06674, Link Cited by: §2.4, §6. Inan 等人 (2023) H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, 和 M. Khabsa Llama Guard: 用于人机对话的基于 LLM 的输入输出安全防护。 外部链接:2312.06674, 链接 引用自:§2.4, §6.
- The task shield: enforcing task alignment to defend against indirect prompt injection in llm agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 29680–29697. Cited by: §1, §1, §2.4, §6. Jia 等人 (2025) F. Jia, T. Wu, X. Qin, 和 A. Squicciarini The task shield: 强制执行任务对齐以防御 LLM 代理中的间接提示词注入。 收录于:第 63 届计算语言学协会年会论文集(第 1 卷:长论文), 第 29680–29697 页。 引用自:§1, §1, §2.4, §6.
- SoK: attack and defense landscape of agentic AI systems. In 35th USENIX Security Symposium (USENIX Security 26), pp. 6047–6066. External Links: Link Cited by: §1, §1, §2.4, §6. Kim 等人 (2026) J. Kim, W. Guo, 和 D. Song SoK: 代理 AI 系统的攻击与防御全景。 收录于:第 35 届 USENIX 安全研讨会 (USENIX Security 26), 第 6047–6066 页。 外部链接:链接 引用自:§1, §1, §2.4, §6.
- An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), External Links: Document, Link Cited by: §6. Larson 等人 (2019) S. Larson, A. Mahendran, J. J. Peper, C. Clarke, A. Lee, P. Hill, J. K. Kummerfeld, K. Leach, M. A. Laurenzano, L. Tang, 和 J. Mars 用于意图分类和领域外预测的评估数据集。 收录于:2019 年自然语言处理实证方法会议暨第 9 届自然语言处理国际联合会议 (EMNLP-IJCNLP) 论文集, 外部链接:文档, 链接 引用自:§6.
- Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 1831–1847. Cited by: §6. Liu 等人 (2024) Y. Liu, Y. Jia, R. Geng, J. Jia, 和 N. Z. Gong 提示词注入攻击与防御的规范化与基准测试。 收录于:第 33 届 USENIX 安全研讨会 (USENIX Security 24), 第 1831–1847 页。 引用自:§6.
- Prompt shields for azure ai content safety. Note: https://learn.microsoft.com/azure/ai-services/content-safety/concepts/jailbreak-detection Cited by: §1, §2.4. Microsoft (2025) Microsoft Azure AI 内容安全提示词防护。 注:https://learn.microsoft.com/azure/ai-services/content-safety/concepts/jailbreak-detection 引用自:§1, §2.4.
- OWASP top 10 for llm applications 2025. Note: https://genai.owasp.org/llm-top-10/Version 2.0 Cited by: §1. OWASP Foundation (2025) OWASP Foundation 2025 年 OWASP LLM 应用十大安全风险。 注:https://genai.owasp.org/llm-top-10/Version 2.0 引用自:§1.
- NeMo Guardrails: a toolkit for controllable and safe LLM applications with programmable rails. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP), pp. 431–445. External Links: Document, Link Cited by: §2.4, §6. Rebedea 等人 (2023) T. Rebedea, R. Dinu, M. N. Sreedhar, C. Parisien, 和 J. Cohen NeMo Guardrails: 具有可编程防护栏的可控且安全的 LLM 应用工具包。 收录于:2023 年自然语言处理实证方法会议 (EMNLP) 系统演示论文集, 第 431–445 页。 外部链接:文档, 链接 引用自:§2.4, §6.
- Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 36. External Links: Link Cited by: §1. Schick 等人 (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, 和 T. Scialom Toolformer: 语言模型可以自学使用工具。 收录于:神经信息处理系统 (NeurIPS) 进展, 第 36 卷。 外部链接:链接 引用自:§1.
- Constitutional classifiers: defending against universal jailbreaks across thousands of hours of red teaming. Note: Safeguards Research Team, Anthropic External Links: 2501.18837, Link Cited by: §6. Sharma 等人 (2025) M. Sharma, M. Tong, J. Mu, J. Wei, J. Kruthoff, S. Goodfriend, E. Ong 等。 宪法分类器:防御数千小时红队测试中的通用越狱攻击。 注:Anthropic 安全防护研究团队 外部链接:2501.18837, 链接 引用自:§6.
- Progent: progent: securing ai agents with privilege control. External Links: 2504.11703, Link Cited by: §6. Shi 等人 (2025) T. Shi, J. He, Z. Wang, H. Li, L. Wu, W. Guo, 和 D. Song Progent: 通过权限控制保障 AI 代理安全。 外部链接:2504.11703, 链接 引用自:§6.
- The instruction hierarchy: training LLMs to prioritize privileged instructions. External Links: 2404.13208, Link Cited by: §6. Wallace 等人 (2024) E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, 和 A. Beutel 指令层级:训练 LLM 优先处理特权指令。 外部链接:2404.13208, 链接 引用自:§6.
- Autogen: enabling next-gen llm applications via multi-agent conversation. arXiv preprint arXiv:2308.08155. Cited by: §1. Wu 等人 (2023) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu 等。 Autogen: 通过多代理对话赋能下一代 LLM 应用。 arXiv 预印本 arXiv:2308.08155。 引用自:§1.
- IsolateGPT: an execution isolation architecture for LLM-based agentic systems. In Proceedings of the Network and Distributed System Security Symposium (NDSS), External Links: Document, Link Cited by: §6. Wu 等人 (2025) Y. Wu, F. Roesner, T. Kohno, N. Zhang, 和 U. Iqbal IsolateGPT: 基于 LLM 的代理系统的执行隔离架构。 收录于:网络与分布式系统安全研讨会 (NDSS) 论文集, 外部链接:文档, 链接 引用自:§6.
- React: synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Cited by: §1. Yao 等人 (2022) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, 和 Y. Cao React: 在语言模型中协同推理与行动。 arXiv 预印本 arXiv:2210.03629。 引用自:§1.
- ShieldGemma: generative AI content moderation based on gemma. External Links: 2407.21772, Link Cited by: §6. Zeng 等人 (2024) W. Zeng, Y. Liu, R. Mullins, L. Peran, J. Fernandez, H. Harkous, K. Narasimhan, D. Proud, P. Kumar, B. Radharapu, O. Sturman, 和 O. Wahltinez ShieldGemma: 基于 Gemma 的生成式 AI 内容审核。 外部链接:2407.21772, 链接 引用自:§6.
- InjecAgent: benchmarking indirect prompt injections in tool-integrated large language model agents. In Findings of the Association for Computational Linguistics (ACL), External Links: Document, Link Cited by: §6. Zhan 等人 (2024) Q. Zhan, Z. Liang, Z. Ying, 和 D. Kang InjecAgent: 对工具集成 LLM 代理中的间接提示词注入进行基准测试。 收录于:计算语言学协会 (ACL) 发现论文集, 外部链接:文档, 链接 引用自:§6.
- Agent security bench (ASB): formalizing and benchmarking attacks and defenses in LLM-based agents. In International Conference on Learning Representations (ICLR), External Links: 2410.02644, Link Cited by: §6. Zhang 等人 (2025) H. Zhang, J. Huang, K. Mei, Y. Yao, Z. Wang, C. Zhan, H. Wang, 和 Y. Zhang Agent security bench (ASB): 对基于 LLM 的代理中的攻击与防御进行规范化与基准测试。 收录于:国际学习表征会议 (ICLR), 外部链接:2410.02644, 链接 引用自:§6.
- MELON: provable defense against indirect prompt injection attacks in AI agents. In Proceedings of the 42nd International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, Vol. 267. External Links: 2502.05174, Link Cited by: §6. Zhu 等人 (2025) K. Zhu, X. Yang, J. Wang, W. Guo, 和 W. Y. Wang MELON: AI 代理中针对间接提示词注入攻击的可证明防御。 收录于:第 42 届国际机器学习会议 (ICML) 论文集, 机器学习研究论文集,第 267 卷。 外部链接:2502.05174, 链接 引用自:§6.
| Decision | Baseline | +Allowlist | +Iterated | Blocklist | Allow+Block |
|---|---|---|---|---|---|
| Correct blocked (TP) | 747 | 897 | 949 | 508 | 976 |
| False blocked (FP) | 37 | 37 | 36 | 87 | 82 |
| Correct approved (TN) | 967 | 967 | 968 | 917 | 922 |
| False approved (FN) | 249 | 99 | 47 | 488 | 20 |
Appendix A Additional Experimental Results附录 A 补充实验结果
This appendix presents additional experimental results that complement the evaluation in Section 4. Table 9 reports the underlying decision counts for each guardrail configuration. Table 10 summarizes the length distribution of messages in the PAGE benchmark, illustrating the diversity of the evaluation corpus. Table 11 reports the performance of the iterated configuration for each of the four evaluated agent domains, demonstrating that Persona Guardrail generalizes consistently across agents. Table 12 validates the iterative allowlist refinement on disjoint held-out splits, confirming the gains are not an artifact of tuning on the evaluation set.本附录提供了补充实验结果,作为第 4 节评估的补充。表 9 报告了每种防护栏配置的底层决策计数。表 10 总结了 PAGE 基准测试中消息的长度分布,展示了评估语料库的多样性。表 11 报告了迭代配置在四个被评估代理领域中的表现,证明了 Persona Guardrail 在不同代理之间具有一致的泛化能力。表 12 验证了在不相交的留出集上进行的迭代允许列表细化,证实了性能提升并非通过在评估集上调优产生的伪影。
A.1. Iterating the allowlist on held-out data.A.1. 在留出数据上迭代允许列表。
To ensure refinement is not overfitting, we partitioned PAGE into disjoint two-way () and three-way () splits and scored iterations on unseen data (Table 12). Testing v1 (initial), v2 (iterated, Table 6), and v3 (refined from v2’s part 2 errors, tested on part 3) showed monotonic accuracy gains on unseen data ( for ; for ). Furthermore, v3 cut false approvals () and boosted out-of-domain detection (), confirming true functional-boundary gains over overfitting.为确保细化过程不存在过拟合,我们将 PAGE 分割为不相交的二分(50/50)和三分(33/33/33)拆分,并在未见数据上对迭代结果进行评分(表 12)。测试 v1(初始)、v2(迭代,表 6)和 v3(根据 v2 的第 2 部分错误进行细化,并在第 3 部分进行测试)显示在未见数据上准确率呈单调增长(50/50 拆分中为 92.4%→96.0%;33/33/33 拆分中为 91.9%→96.1%→96.7%)。此外,v3 减少了误批准(5.1%→2.7%)并提升了领域外检测率(93.9%→95.8%),证实了功能边界的真实提升而非过拟合。
A.2. Defense against informed attackersA.2. 针对知情攻击者的防御
In additional experiments, we tested how Persona Guard withstands automated attackers possessing varying levels of system knowledge. We evaluated several models as attackers (Opus 4.8, Opus 5, Sonnet 5, Luna, Fable 5, and GPT-5.5) across three knowledge tiers: general architecture, the exact model and prompt structure, and the verbatim allowed intent lists. Attackers were granted multiple turns to bypass the guardrail if their initial attempts failed. We observed that access to the verbatim intent list provided the greatest advantage to the attacker, a scenario that carries relatively low real-world risk. Conversely, knowing the exact underlying model offered no additional benefit once the attacker was already aware of the guardrail’s presence. Even in the worst-case scenario, where the attacker had full access to the intent list, the system blocked over 85% of attacks generated by the most capable model, while maintaining a 90%+ detection rate across all other scenarios. A notable limitation during these experiments was the safety tuning of the closed-source models used as attackers. Because these models frequently refused to generate adversarial prompts, we observed a refusal rate between 17% and 71% on first-turn attacks.在补充实验中,我们测试了 Persona Guard 如何抵御具备不同系统知识水平的自动化攻击者。我们评估了多个模型作为攻击者(Opus 4.8、Opus 5、Sonnet 5、Luna、Fable 5 和 GPT-5.5),涵盖三个知识层级:通用架构、确切的模型和提示词结构,以及逐字的允许意图列表。如果初始尝试失败,攻击者将被授予多轮机会以绕过防护栏。我们观察到,获取逐字意图列表为攻击者提供了最大的优势,而这种情况在现实世界中风险相对较低。相反,一旦攻击者意识到防护栏的存在,了解确切的底层模型并不会提供额外的好处。即使在最坏的情况下(即攻击者完全访问意图列表),系统仍能拦截超过 85% 由最强模型生成的攻击,同时在所有其他场景中保持 90% 以上的检测率。这些实验中的一个显著局限是所使用的闭源攻击模型的安全调优。由于这些模型经常拒绝生成对抗性提示词,我们在首轮攻击中观察到了 17% 到 71% 的拒绝率。
Appendix B Distribution of Production Traffic附录 B 生产流量分布
As Figure 4 shows, most production guardrail prompts are short (under 1,000 tokens). The small fraction of requests that exceed the context limit are rejected before inference to protect latency and GPU utilization.如图 4 所示,大多数生产环境的防护栏提示词都很短(少于 1,000 个 token)。极少数超过上下文限制的请求会在推理前被拒绝,以保护延迟和 GPU 利用率。
| Request kind | Median | Mean | P90 | Max |
|---|---|---|---|---|
| Benign | 17 | 20.7 | 33 | 121 |
| Adversarial | 14 | 15.4 | 22 | 84 |
| Out-of-domain | 23 | 30.9 | 63 | 121 |
| Mixed | 37 | 44.1 | 68 | 152 |
| Overall | 17 | 23.8 | 47 | 152 |
| Agent | FBR | Adv. det. | OOD det. |
|---|---|---|---|
| Workspace | 3.1 | 97.8 | 92.6 |
| Slack | 4.0 | 98.4 | 85.6 |
| Banking | 6.4 | 93.4 | 98.4 |
| Travel | 0.8 | 98.3 | 97.6 |
| Average | 3.6 | 97.0 | 93.5 |
| Iteration (split, part) | Acc | FAR | FBR | Adv | OOD |
| v1 (, p1) | 92.4 | 11.0 | 4.2 | 96.5 | 81.5 |
| v2 (, p2) | 96.0 | 4.8 | 3.2 | 96.7 | 93.6 |
| v1 (, p1) | 91.9 | 11.5 | 4.8 | 95.8 | 81.2 |
| v2 (, p2) | 96.1 | 5.1 | 2.7 | 95.8 | 93.9 |
| v3 (, p3) | 96.7 | 2.7 | 3.9 | 98.8 | 95.8 |
Appendix C Open Science附录 C 开放科学
We provide the following artifacts to support evaluation and reproduction of the paper’s core results:我们提供以下工件以支持本文核心结果的评估和复现:
- •
The PAGE benchmark, including the user- and agent-turn examples, persona and request-kind labels, and evaluation splits; andPAGE 基准测试,包括用户和代理轮次示例、角色和请求类型标签以及评估拆分;以及
- •
The agent-specific intent specifications used in the experiments, including both allowed and blocked intents for the four evaluated domains.实验中使用的特定于代理的意图规范,包括四个被评估领域中允许和拦截的意图。
The artifacts are available through the following URL: https://github.com/uber-research/page-benchmark.这些工件可通过以下 URL 获取: https://github.com/uber-research/page-benchmark。
We cannot release the exact production guardrail prompt due to security risks and proprietary details. Instead, the paper provides the methodology, model details, and intent specifications so researchers can build a comparable classifier. While exact reproduction isn’t possible without our internal infrastructure, these tools allow researchers to achieve similar results.由于安全风险和专有细节,我们无法发布确切的生产防护栏提示词。相反,本文提供了方法论、模型细节和意图规范,以便研究人员可以构建可比的分类器。虽然没有我们的内部基础设施无法进行精确复现,但这些工具允许研究人员获得类似的结果。
Appendix D Use of Generative AI附录 D 生成式 AI 的使用
We used generative AI tools during this work for software development assistance (including code generation and refactoring), generation of synthetic benchmark examples and prompt templates, and editorial assistance for improving clarity, grammar, and organization of the manuscript. All technical designs, implementation decisions, experimental methodology, benchmark construction, evaluation, and scientific conclusions were developed, verified, and approved by the authors. All AI-generated code, text, and synthetic data were reviewed and, where appropriate, modified prior to inclusion in the final artifact.我们在本研究中使用了生成式 AI 工具来辅助软件开发(包括代码生成和重构)、生成合成基准示例和提示词模板,以及提供编辑协助以改善文稿的清晰度、语法和组织结构。所有技术设计、实现决策、实验方法、基准测试构建、评估和科学结论均由作者开发、验证和批准。所有 AI 生成的代码、文本和合成数据在纳入最终工件之前均经过审查,并在适当情况下进行了修改。