Federated Agent Optimization联邦智能体优化

Qiang Yang Affiliation: The Hong Kong Polytechnic University, Hong Kong SAR, China Zhiqiang Kou Affiliation: The Hong Kong Polytechnic University, Hong Kong SAR, China Xueyi Zhang Affiliation: National University of Singapore, Singapore Dong-Dong Wu Affiliation: The University of Tokyo, Japan Hanlin Gu Affiliation: Hong Kong University of Science and Technology, Hong Kong SAR, China^*Corresponding author: profqiang.yang@polyu.edu.hk Jing Guo Affiliation: The Hong Kong Polytechnic University, Hong Kong SAR, China Yang Liu Affiliation: The Hong Kong Polytechnic University, Hong Kong SAR, China Di Jiang Affiliation: The Hong Kong Polytechnic University, Hong Kong SAR, China Qian Xu Affiliation: Tengen AI, Hong Kong SAR, China

Abstract摘要

Large language model (LLM) agents increasingly operate in private environments and accumulate valuable experience from task execution, tool use, feedback, and local knowledge. Yet such experience is distributed across organizations and cannot be directly shared because of privacy and proprietary constraints. Conventional federated learning is insufficient for this setting, as agent capabilities extend beyond model parameters to memory, tools, rewards, skills, and structured knowledge. In this paper, we formulate Federated Agent Optimization (FAO), which studies how distributed agents can collaboratively improve through controlled information exchange while keeping raw data, complete trajectories, and private knowledge local. We define FAO as a multi-objective problem balancing agent utility, privacy leakage, and communication cost, and organize its optimization space across policy, memory, tool use, reward, and structured knowledge and skills. We further characterize how private experience can be abstracted, protected, aggregated, and adapted into transferable capabilities, providing a unified view of how agents can benefit from one another without direct experience sharing. Finally, we identify the key challenges of FAO and outline several promising directions for future research toward trustworthy federated agent systems.大型语言模型（LLM）智能体越来越多地在私有环境中运行，并从任务执行、工具使用、反馈和本地知识中积累宝贵的经验。然而，这些经验分布在各个组织中，由于隐私和所有权限制，无法直接共享。传统的联邦学习不足以应对这一场景，因为智能体的能力不仅限于模型参数，还扩展到记忆、工具、奖励、技能和结构化知识。在本文中，我们提出了联邦智能体优化（Federated Agent Optimization, FAO），旨在研究分布式智能体如何在保持原始数据、完整轨迹和私有知识本地化的前提下，通过受控的信息交换进行协作改进。我们将 FAO 定义为一个平衡智能体效用、隐私泄露和通信成本的多目标问题，并将其优化空间组织在策略、记忆、工具使用、奖励以及结构化知识和技能方面。我们进一步描述了如何将私有经验抽象、保护、聚合并转化为可迁移的能力，从而提供了一个统一的视角，展示智能体如何在不直接共享经验的情况下相互受益。最后，我们确定了 FAO 的关键挑战，并为未来构建可信的联邦智能体系统指明了几个有前景的研究方向。

1. Introduction1. 引言

In recent years, large language model (LLM)-driven agents [60, 130, 182] have gradually evolved from general-purpose conversational systems [5] toward executing complex tasks in real-world environments, becoming an important paradigm for enterprise AI systems [207]. An increasing number of organizations are deploying agents in applications such as customer service [178, 58, 66], software development [146, 19], data analysis [72, 155], knowledge management [177, 197, 160], office automation [191], financial services [158], healthcare assistance [142, 125, 79, 97], and business process execution [101, 81, 65, 40]. Unlike traditional language models that primarily focus on content generation [102], enterprise agents [135] typically need to integrate internal knowledge [78], business data, proprietary tools, and operational workflows to autonomously perform task planning [143, 190], tool invocation [36], environment interaction [13, 28], and outcome evaluation [248]. As their applications continue to expand, agents are evolving from standalone language generation models [45] into complex intelligent systems [236] that can perceive their environments, take actions, and continuously adapt to changing business requirements.近年来，以大型语言模型（LLM）为驱动的智能体 [60, 130, 182] 已逐渐从通用对话系统 [5] 演变为在现实环境中执行复杂任务的系统，成为企业 AI 系统的重要范式 [207]。越来越多的组织正在将智能体部署在客户服务 [178, 58, 66]、软件开发 [146, 19]、数据分析 [72, 155]、知识管理 [177, 197, 160]、办公自动化 [191]、金融服务 [158]、医疗辅助 [142, 125, 79, 97] 以及业务流程执行 [101, 81, 65, 40] 等应用中。与主要关注内容生成的传统语言模型 [102] 不同，企业智能体 [135] 通常需要整合内部知识 [78]、业务数据、专有工具和操作流程，以自主执行任务规划 [143, 190]、工具调用 [36]、环境交互 [13, 28] 和结果评估 [248]。随着其应用范围的不断扩大，智能体正从独立的语言生成模型 [45] 演变为复杂的智能系统 [236]，能够感知环境、采取行动并持续适应不断变化的业务需求。

During continuous task execution, enterprise agents generate large volumes of new workflows, execution trajectories [12, 63], tool-use records [188, 46], and success or failure feedback on a daily basis [141]. These real-world interaction experiences can help agents continuously refine task planning [143, 190], tool utilization, and decision-making strategies [159], thereby improving their practical task performance over time. Meanwhile, as agents are deployed across different organizations [42, 249], departments, and user environments, valuable experience becomes increasingly distributed across isolated local systems. Different agents often face similar or related tasks, and their accumulated successful experiences, failure patterns, and tool-use strategies are naturally complementary, creating substantial potential for mutual improvement [105]. However, such experience is often tightly coupled with user data, internal knowledge, business rules, proprietary tools, and operational workflows, making it difficult to directly share raw data or complete execution trajectories [37]. As a result, large amounts of potentially reusable agent experience remain confined to local environments, limiting effective collaboration among agents and preventing the full value of cross-environment experience from being realized.在持续的任务执行过程中，企业智能体每天都会产生大量的新工作流、执行轨迹 [12, 63]、工具使用记录 [188, 46] 以及成功或失败的反馈 [141]。这些现实世界的交互经验可以帮助智能体不断优化任务规划 [143, 190]、工具利用和决策策略 [159]，从而随时间推移提高其实际任务性能。与此同时，随着智能体部署在不同的组织 [42, 249]、部门和用户环境中，宝贵的经验日益分散在孤立的本地系统中。不同的智能体往往面临相似或相关的任务，它们积累的成功经验、失败模式和工具使用策略具有天然的互补性，为相互改进创造了巨大的潜力 [105]。然而，这些经验往往与用户数据、内部知识、业务规则、专有工具和操作流程紧密耦合，使得直接共享原始数据或完整的执行轨迹变得困难 [37]。结果，大量潜在可重用的智能体经验被局限在本地环境中，限制了智能体之间的有效协作，并阻碍了跨环境经验价值的充分实现。

Federated learning [196, 103, 226] provides a natural technical foundation for mitigating the experience isolation described above. It enables multiple clients [55] to collaboratively learn by exchanging and aggregating model updates while keeping raw data local [41], thereby avoiding the direct centralization of sensitive information. However, traditional federated learning primarily focuses on model parameters, whereas the capabilities of enterprise agents are determined not only by the underlying model but also by their decision policies, memories, tool use, environmental feedback, and local knowledge and business rules. Therefore, in agent settings [7], parameter aggregation alone is insufficient to enable effective cross-agent collaboration. A more fundamental question is how to transform the useful experience, skills, and feedback acquired by one agent during real-world execution into shareable capabilities that can benefit other agents without exposing local private information. Existing federated learning paradigms have not yet systematically addressed this problem and still lack a unified optimization framework for the broader capabilities of agents.联邦学习 [196, 103, 226] 为缓解上述经验孤岛问题提供了天然的技术基础。它使多个客户端 [55] 能够在保持原始数据本地化的同时，通过交换和聚合模型更新进行协作学习 [41]，从而避免了敏感信息的直接集中。然而，传统的联邦学习主要关注模型参数，而企业智能体的能力不仅由底层模型决定，还由其决策策略、记忆、工具使用、环境反馈以及本地知识和业务规则决定。因此，在智能体场景 [7] 中，仅靠参数聚合不足以实现有效的跨智能体协作。一个更根本的问题是，如何将一个智能体在实际执行过程中获得的有用经验、技能和反馈转化为可共享的能力，在不暴露本地私有信息的情况下造福其他智能体。现有的联邦学习范式尚未系统地解决这一问题，且仍缺乏针对智能体更广泛能力的统一优化框架。

We refer to this class of problems as Federated Agent Optimization (FAO). FAO studies how multiple agents operating in different private environments can collaboratively improve through controlled information exchange while keeping raw data, complete interaction trajectories, and local private knowledge decentralized. Unlike conventional federated learning, which primarily optimizes model parameters, FAO considers the broader capabilities of an agent, including its policy, memory, tool use, reward mechanism, and structured knowledge. We formulate FAO around three interrelated objectives: improving local agent utility, reducing privacy leakage during collaboration, and controlling communication and system costs. In addition, effective federation should provide meaningful collaboration gains to individual participants rather than improving only aggregate performance.我们将这类问题称为联邦智能体优化（FAO）。FAO 研究在不同私有环境中运行的多个智能体如何通过受控的信息交换进行协作改进，同时保持原始数据、完整交互轨迹和本地私有知识的去中心化。与主要优化模型参数的传统联邦学习不同，FAO 考虑了智能体的更广泛能力，包括其策略、记忆、工具使用、奖励机制和结构化知识。我们围绕三个相互关联的目标构建 FAO：提高本地智能体效用、减少协作过程中的隐私泄露，以及控制通信和系统成本。此外，有效的联邦化应为单个参与者提供有意义的协作收益，而不仅仅是提高聚合性能。

In this paper, we establish a unified framework for studying FAO. First, we formally define local agents, private environments, federated protocols, information-sharing boundaries, and a multi-objective optimization formulation that jointly captures performance, privacy, and communication cost. Second, we organize existing and emerging approaches according to the agent capability being federated, including policies, memory, tool use, rewards, and structured knowledge and skills. Third, we characterize how private local experience can be transformed into transferable capabilities through abstraction, protection, aggregation, and local adaptation, and further examine when such transfer leads to mutual improvement across agents. Building on this framework, we identify key challenges in balancing performance, privacy, and communication, ensuring fair contribution and benefit allocation, and reliably evaluating federation gains. Finally, we highlight several open research directions toward trustworthy, efficient, and continuously improving federated agent systems.在本文中，我们建立了一个研究 FAO 的统一框架。首先，我们正式定义了本地智能体、私有环境、联邦协议、信息共享边界，以及一个共同捕捉性能、隐私和通信成本的多目标优化公式。其次，我们根据被联邦化的智能体能力（包括策略、记忆、工具使用、奖励以及结构化知识和技能）对现有和新兴方法进行了分类。第三，我们描述了私有本地经验如何通过抽象、保护、聚合和本地适应转化为可迁移的能力，并进一步探讨了这种迁移在何时能带来智能体间的相互改进。基于该框架，我们确定了在平衡性能、隐私和通信、确保公平贡献和利益分配以及可靠评估联邦收益方面所面临的关键挑战。最后，我们强调了迈向可信、高效且持续改进的联邦智能体系统的几个开放研究方向。

Paper Roadmap.论文路线图。

The remainder of this paper is organized as follows. Section 2 reviews the most relevant research areas. Section 3 formally defines Federated Agent Optimization (FAO). Section 4 presents a taxonomy of FAO according to the agent components being collaboratively optimized. Section 5 further investigates how local experience can be transformed into transferable capabilities. Section 6 summarizes the key challenges of FAO. Finally, Section 7 discusses future works.本文其余部分的组织结构如下。第 2 节回顾了最相关的研究领域。第 3 节正式定义了联邦智能体优化（FAO）。第 4 节根据协作优化的智能体组件对 FAO 进行了分类。第 5 节进一步研究了本地经验如何转化为可迁移的能力。第 6 节总结了 FAO 的关键挑战。最后，第 7 节讨论了未来的工作。

2. Related Work2. 相关工作

Federated Learning and Federated Large Models联邦学习与联邦大模型

Federated learning [196, 103, 226] enables multiple clients [55] to collaboratively train models while keeping raw data local, providing a general paradigm for privacy-preserving distributed learning [80]. Early studies [145] mainly focused on model aggregation under heterogeneous data distributions, while subsequent work further explored communication efficiency [33], personalization [173], robustness [76, 107], and privacy protection [195, 106]. With the emergence of foundation models and large language models [241, 126], federated learning has been extended to large-scale model adaptation [93], where parameter-efficient techniques such as federated fine-tuning [193, 198], LoRA [171, 14], prompt tuning [239, 27], and knowledge distillation [162] are increasingly adopted to reduce computation and communication costs. Despite these advances, existing federated large-model methods still primarily treat model parameters or representations as the main optimization objects. In contrast, an agent [60] typically involves not only an underlying model, but also memory, tool use, rewards, skills, and external knowledge. Therefore, federating a large model is not equivalent to federating an agent, motivating a broader optimization paradigm that considers the complete capabilities of agents.联邦学习 [196, 103, 226] 使多个客户端 [55] 能够在保持原始数据本地化的同时协作训练模型，为隐私保护的分布式学习提供了一种通用范式 [80]。早期研究 [145] 主要关注异构数据分布下的模型聚合，后续工作进一步探索了通信效率 [33]、个性化 [173]、鲁棒性 [76, 107] 和隐私保护 [195, 106]。随着基础模型和大型语言模型 [241, 126] 的出现，联邦学习已扩展到大规模模型适应 [93]，其中联邦微调 [193, 198]、LoRA [171, 14]、提示词微调 [239, 27] 和知识蒸馏 [162] 等参数高效技术被越来越多地采用，以降低计算和通信成本。尽管取得了这些进展，现有的联邦大模型方法仍主要将模型参数或表示视为主要的优化对象。相比之下，智能体 [60] 通常不仅涉及底层模型，还涉及记忆、工具使用、奖励、技能和外部知识。因此，联邦化一个大模型并不等同于联邦化一个智能体，这促使我们需要一个考虑智能体完整能力的更广泛的优化范式。

Multi-Agent Systems and Agent Reinforcement learning多智能体系统与智能体强化学习

Multi-agent systems [69, 221, 98] study how multiple agents coordinate, communicate, divide tasks, and make joint decisions in shared or interacting environments. With the rise of large language models, recent LLM-based multi-agent systems [69, 210, 34] have further explored role specialization, discussion, negotiation, and mutual feedback to improve performance on complex tasks. In parallel, agent learning focuses on continuously improving agent capabilities through execution trajectories [176], environmental feedback [219], reinforcement learning [245, 227], self-reflection [17], and teacher-agent guidance [215]. Despite this progress, most existing multi-agent and agent-learning methods assume that agents can directly exchange messages, trajectories, or feedback during collaboration, while privacy constraints and data isolation across organizations or private environments are rarely considered. This differs fundamentally from federated agent optimization, where agents are expected to benefit from each other while keeping local data, execution experience, and private knowledge decentralized.多智能体系统 [69, 221, 98] 研究多个智能体如何在共享或交互环境中协调、通信、分工和共同决策。随着大型语言模型的兴起，近期的基于 LLM 的多智能体系统 [69, 210, 34] 进一步探索了角色专业化、讨论、协商和相互反馈，以提高复杂任务的性能。与此同时，智能体学习专注于通过执行轨迹 [176]、环境反馈 [219]、强化学习 [245, 227]、自我反思 [17] 和教师智能体指导 [215] 来持续改进智能体能力。尽管取得了这些进展，但大多数现有的多智能体和智能体学习方法都假设智能体可以在协作过程中直接交换消息、轨迹或反馈，而很少考虑跨组织或私有环境的隐私约束和数据孤岛问题。这与联邦智能体优化有本质区别，在 FAO 中，智能体在保持本地数据、执行经验和私有知识去中心化的同时，预期能从彼此身上获益。

Privacy-Preserving AI隐私保护 AI

Privacy-preserving AI [131] aims to reduce the risk of sensitive information leakage while enabling effective model training and inference. Common techniques include differential privacy [6], secure aggregation [118], homomorphic encryption [87], secure multi-party computation [161], and access control [83]. These techniques have been widely adopted in distributed learning [32, 22], cloud-based inference [62], and cross-organization collaboration [26, 132] to protect training data, model updates, and intermediate representations. However, existing privacy-preserving methods mainly focus on data, model parameters, or inference processes. In agent systems, sensitive information may also be embedded in execution trajectories, memories, tool interactions, business rules, and skill representations, which calls for new protection mechanisms tailored to collaborative agent optimization.隐私保护 AI [131] 旨在降低敏感信息泄露的风险，同时实现有效的模型训练和推理。常用技术包括差分隐私 [6]、安全聚合 [118]、同态加密 [87]、安全多方计算 [161] 和访问控制 [83]。这些技术已被广泛应用于分布式学习 [32, 22]、基于云的推理 [62] 和跨组织协作 [26, 132]，以保护训练数据、模型更新和中间表示。然而，现有的隐私保护方法主要关注数据、模型参数或推理过程。在智能体系统中，敏感信息也可能嵌入在执行轨迹、记忆、工具交互、业务规则和技能表示中，这需要针对协作智能体优化定制新的保护机制。

3. Problem Definition: Federated Agent Optimization3. 问题定义：联邦智能体优化

We formalize Federated Agent Optimization (FAO) for $K$ clients in the horizontal setting, with the notation of Appendix A. All clients draw their tasks from one task space $\mathcal{X}$ , and client $k$ has a task distribution $\mathcal{Q}_{k}$ , a utility $u_{k}$ , and a private dataset $\mathcal{D}_{k}$ of $n_{k}$ tasks with the trajectories logged by its earlier agents,我们对水平设置下 K 个客户端的联邦智能体优化（FAO）进行了形式化，符号说明见附录 A。所有客户端从同一个任务空间 𝒳 中抽取任务，客户端 k 拥有任务分布 𝒬_k、效用 u_k 以及包含 n_k 个任务的私有数据集 𝒟_k，这些任务的轨迹由其先前的智能体记录。

\operatorname{supp}(\mathcal{Q}_{k})\subseteq\mathcal{X},\qquad\mathcal{D}_{k}=\big\{(x_{k,i},\tau_{k,i})\big\}_{i=1}^{n_{k}},\quad x_{k,i}\sim\mathcal{Q}_{k},\qquad k=1,\dots,K.

(1)

The distributions $\mathcal{Q}_{k}$ and utilities $u_{k}$ may differ across clients and coincide in the homogeneous case. $\mathcal{D}_{k}$ also holds the raw records from which the agent builds its memory and knowledge, and none of it may leave the client.分布 𝒬_k 和效用 u_k 在不同客户端之间可能不同，在同构情况下则一致。𝒟_k 还保存了智能体构建其记忆和知识所依据的原始记录，且这些记录均不得离开客户端。

3.1 Local Agents and Private Data3.1 本地智能体与私有数据

Client $k$ starts from an initial agent $A_{k}^{0}$ , and each of its agents has five components [169],客户端 k 从初始智能体 A_k^0 开始，其每个智能体包含五个组件 [169]：

A_{k}=\big(\underbrace{\pi_{k}}_{\text{policy}},\ \underbrace{\mathcal{M}_{k}}_{\text{memory}},\ \underbrace{\mathcal{T}_{k}}_{\text{tools}},\ \underbrace{\mathcal{R}_{k}}_{\text{reward}},\ \underbrace{\mathcal{O}_{k}}_{\text{structured knowledge}}\big).

(2)

The policy $\pi_{k}=\pi_{\theta_{k}}$ maps a context to an action, and $\theta_{k}$ is continuous (weights, adapters) or discrete (prompts, workflow graphs). The memory $\mathcal{M}_{k}$ stores experience for retrieval [167, 237, 234], $\mathcal{T}_{k}$ is the set of callable tools with their descriptions, and $\mathcal{R}_{k}$ is the local evaluator. $\mathcal{O}_{k}$ holds curated structured knowledge, namely ontologies, rules on the business and the environment, and skills, which, unlike memory items, are stated explicitly rather than retrieved by similarity.策略 π_k = π_{θ_k} 将上下文映射为动作，θ_k 可以是连续的（权重、适配器）或离散的（提示词、工作流图）。记忆 ℳ_k 存储用于检索的经验 [167, 237, 234]，𝒯_k 是带有描述的可调用工具集合，ℛ_k 是本地评估器。𝒪_k 保存策划好的结构化知识，即本体、业务和环境规则以及技能，与记忆项不同，这些知识是明确陈述而非通过相似度检索的。

On a task $x$ , an agent $A$ produces a trajectory $\tau$ of $H$ steps, and its performance on client $k$ is the expected utility under the task distribution of the client,在任务 x 上，智能体 A 产生一个 H 步的轨迹 τ，其在客户端 k 上的性能是该客户端任务分布下的期望效用：

	$\displaystyle\tau$	$\displaystyle=(s_{h},a_{h},o_{h},r_{h})_{h=1}^{H}\sim p_{A}(\cdot\mid x),$		(3)
	$\displaystyle U_{k}(A;\mathcal{Q}_{k})$	$\displaystyle=\mathbb{E}_{x\sim\mathcal{Q}_{k}}\,\mathbb{E}_{\tau\sim p_{A}(\cdot\mid x)}\big[u_{k}(\tau)\big],\qquad u_{k}(\tau)\in[0,1],\qquad\mathcal{R}_{k}(\tau)\approx u_{k}(\tau).$		(4)

Here $s_{h}$ is the context, $a_{h}$ the action (a message or a tool call), $o_{h}$ the observation, and $r_{h}$ the feedback, which is often given only at the last step. Since $u_{k}$ is observed only through outcomes and human feedback, the agent is optimized against the estimate $\mathcal{R}_{k}$ .此处 s_h 是上下文，a_h 是动作（消息或工具调用），o_h 是观测值，r_h 是反馈（通常仅在最后一步给出）。由于 u_k 仅通过结果和人类反馈观察到，智能体是针对估计值 ℛ_k 进行优化的。

3.2 Federated Protocol3.2 联邦协议

A coordinator runs a protocol $\Phi$ that fixes the participants $\mathcal{S}^{t}$ of each round, the rules $\Update$ , $\Agg$ , and $\Adapt$ , the protection of the uploads, and the schedule of messages. Round $t=0,\dots,T-1$ consists of a local update, an aggregation, and a local adaptation,协调器运行协议 Φ，该协议确定每一轮的参与者 𝒮^t、规则 Update、Agg 和 Adapt、上传内容的保护方式以及消息调度。第 t 轮（t=0,…,T-1）由本地更新、聚合和本地适应组成：

$\displaystyle\Delta_{k}^{t}$	$\displaystyle=\Update\big(A_{k}^{t},\mathcal{D}_{k};\Phi\big)=\big(\Delta_{k}^{\pi},\,\Delta_{k}^{\mathcal{M}},\,\Delta_{k}^{\mathcal{T}},\,\Delta_{k}^{\mathcal{R}},\,\Delta_{k}^{\mathcal{O}}\big)^{t},\qquad k\in\mathcal{S}^{t},$	(5)
$\displaystyle G^{t+1}$	$\displaystyle=\Agg\big(\{\Delta_{k}^{t}\}_{k\in\mathcal{S}^{t}},\,G^{t};\Phi\big)=\big\{G_{k}^{t+1}\big\}_{k=1}^{K},\qquad G_{k}=\big(G_{k}^{\pi},G_{k}^{\mathcal{M}},G_{k}^{\mathcal{T}},G_{k}^{\mathcal{R}},G_{k}^{\mathcal{O}}\big),$	(6)
$\displaystyle A_{k}^{t+1}$	$\displaystyle=\Adapt\big(A_{k}^{t},G_{k}^{t+1};\mathcal{D}_{k}\big),\qquad A_{k}^{+}=A_{k}^{T},\qquad A_{k}^{\mathrm{loc}}=A_{k}^{T}\ \text{ under }\ G_{k}^{t}=\emptyset\ \ \forall t.$	(7)

The update is typed: it has one block for each component, any block may be empty, and nothing else leaves the client. Typed artifacts were proposed for tool-routing knowledge [25], and Eq. (5) extends the idea to all components. Federated averaging gives all clients one view, $G_{k}=G$ , whereas personalized methods give each client its own view [75, 209]; Appendix D covers protocols without a coordinator. We drop the index $t$ where possible and write $\Delta_{k}$ for all uploads of client $k$ , $A_{k}^{+}$ for its final agent, and $A_{k}^{\mathrm{loc}}$ for the agent that it obtains without the federation (Figure 1).更新是分类型的：每个组件对应一个块，任何块都可以为空，且没有其他信息离开客户端。分类型工件曾被提议用于工具路由知识 [25]，式 (5) 将这一思想扩展到了所有组件。联邦平均为所有客户端提供一个视图 G_k = G，而个性化方法则为每个客户端提供其自己的视图 [75, 209]；附录 D 涵盖了无需协调器的协议。我们在可能的情况下省略索引 t，并用 Δ_k 表示客户端 k 的所有上传内容，A_k^+ 表示其最终智能体，A_k^loc 表示其在没有联邦的情况下获得的智能体（图 1）。

Figure 1: One round of FAO. Clients upload typed updates

\Delta_{k}

, the coordinator aggregates them into views

G_{k}

, and each client adapts its view. Utility is measured at

A_{k}^{+}

, leakage on

\Delta_{k}

, and communication cost on every message that crosses the line.图 1：FAO 的一轮操作。客户端上传分类型的更新 Δ_k，协调器将其聚合为视图 G_k，每个客户端对其视图进行调整。效用在 A_k^+ 上测量，泄露在 Δ_k 上测量，通信成本在跨越界线的每条消息上测量。

3.3 Multi-Objective Formulation3.3 多目标公式

FAO pursues three objectives: performance, measured for each client on its own tasks so that personalization needs no separate term, privacy, and communication, whose cost counts every message of the protocol by its size $|\cdot|$ ,FAO 追求三个目标：性能（在每个客户端上针对其自身任务进行测量，因此个性化无需单独的项）、隐私和通信（其成本按协议中每条消息的大小 |·| 计算）：

\mathcal{C}_{\mathrm{comm}}\big(\{\Delta_{k}\}_{k=1}^{K},\Phi\big)=\sum_{t=0}^{T-1}\sum_{k\in\mathcal{S}^{t}}\Big(\big|\Delta_{k}^{t}\big|+\big|G_{k}^{t+1}\big|\Big).

(8)

Leakage is a property of the randomized mechanism $M_{k}$ that maps $\mathcal{D}_{k}$ to the transcript of uploads, measured by a differential privacy guarantee [47, 1] or by the advantage of the strongest attack in a stated threat model $\mathfrak{A}$ over attacks without uploads,泄露是随机机制 M_k 的一个属性，该机制将 𝒟_k 映射为上传记录，通过差分隐私保证 [47, 1] 或通过在既定威胁模型 𝔄 下最强攻击相对于无上传攻击的优势来衡量：

$\displaystyle\text{transcript:}$	$\displaystyle M_{k}(\mathcal{D}_{k})=\big(\Delta_{k}^{0},\dots,\Delta_{k}^{T-1}\big),$	(9)
$\displaystyle\text{guarantee:}$	$\displaystyle\mathcal{P}_{k}=\inf\big\{\epsilon:\Pr[M_{k}(\mathcal{D})\in S]\leq e^{\epsilon}\Pr[M_{k}(\mathcal{D}^{\prime})\in S]+\delta\ \ \forall S,\ \mathcal{D}\simeq\mathcal{D}^{\prime}\big\},$
$\displaystyle\text{attack:}$	$\displaystyle\mathcal{P}_{k}=\max_{\mathcal{A}\in\mathfrak{A}}\Pr\big[\mathrm{hit}\big(\mathcal{A}(M_{k}(\mathcal{D}_{k}))\big)\big]-\max_{\mathcal{A}\in\mathfrak{A}}\Pr\big[\mathrm{hit}\big(\mathcal{A}(\bot)\big)\big],$
$\displaystyle\text{hit:}$	$\displaystyle\mathrm{hit}(y)\iff\exists\,r\in\mathcal{D}_{k}:\mathrm{sim}(y,r)\geq\gamma,$

where $\simeq$ is one fixed adjacency relation, such as differing in one trajectory, $\delta$ is fixed, $\bot$ stands for no upload, and a hit recovers a record up to similarity $\gamma$ . The guarantee covers all $T$ rounds and hence their composition [47], and the advantage measures what the uploads add to one stated threat. The two notions are not comparable, so an instantiation fixes one scalar metric, written $\mathcal{P}_{k}(M_{k})$ or $\mathcal{P}_{k}(\Phi)$ , and adds further notions as objectives or constraints. Gradients [246, 139], embeddings [128], memory [179], and text written by language models [166] can all leak private data.其中 ≃ 是一个固定的邻接关系（例如在一条轨迹上有所不同），δ 是固定的，⊥ 代表无上传，命中则恢复出相似度为 γ 的记录。该保证涵盖所有 T 轮，因此涵盖其组合 [47]，优势衡量了上传内容对既定威胁的增加量。这两个概念不可比，因此实例化会固定一个标量度量（记为 𝒫_k(M_k) 或 𝒫_k(Φ)），并将其他概念作为目标或约束添加。梯度 [246, 139]、嵌入 [128]、记忆 [179] 以及语言模型编写的文本 [166] 都可能泄露私有数据。

Definition 1 (Federated Agent Optimization).定义 1（联邦智能体优化）。

Given $K$ clients with initial agents $\{A_{k}^{0}\}$ , task distributions $\{\mathcal{Q}_{k}\}$ , private data $\{\mathcal{D}_{k}\}$ , and utilities $\{u_{k}\}$ , FAO is the following multi-objective problem over the protocol $\Phi$ of the federation,给定 K 个具有初始智能体 {A_k^0}、任务分布 {𝒬_k}、私有数据 {𝒟_k} 和效用 {u_k} 的客户端，FAO 是关于联邦协议 Φ 的以下多目标问题：

\max_{\Phi}\ \Bigg[\ \underbrace{\frac{1}{K}\sum_{k=1}^{K}U_{k}\big(A_{k}^{+};\mathcal{Q}_{k}\big)}_{\text{local agent performance}},\quad\underbrace{-\frac{1}{K}\sum_{k=1}^{K}\mathcal{P}_{k}\big(M_{k}\big)}_{\text{local data leakage risk}},\quad\underbrace{-\,\mathcal{C}_{\mathrm{comm}}\big(\{\Delta_{k}\}_{k=1}^{K},\Phi\big)}_{\text{system communication cost}}\ \Bigg],

(10)

where the final agents $A_{k}^{+}$ and the uploads $\Delta_{k}$ result from $A_{k}^{0}$ , $\mathcal{D}_{k}$ , and $\Phi$ through Eqs. (5)–(7), and $F=(F_{1},F_{2},F_{3})$ denotes the three objectives.其中最终智能体 A_k^+ 和上传内容 Δ_k 由 A_k^0、𝒟_k 和 Φ 通过式 (5)–(7) 产生，F=(F_1, F_2, F_3) 表示这三个目标。

FAO shares its objectives with constrained multi-objective federated learning [85] and differs from it in three properties of its decision space. First, each block of an update is a parameter vector, a text, or a structure such as a graph or a rule set,FAO 与受约束的多目标联邦学习 [85] 共享目标，但在决策空间的三个属性上有所不同。首先，更新的每个块都是一个参数向量、文本或结构（如图形或规则集）：

\Delta_{k}^{c}\ \in\ \underbrace{\mathbb{R}^{d_{c}}}_{\text{parameters}}\ \cup\ \underbrace{\Sigma^{*}}_{\text{text}}\ \cup\ \underbrace{\mathcal{G}}_{\text{structures}},\qquad c\in\{\pi,\mathcal{M},\mathcal{T},\mathcal{R},\mathcal{O}\},

(11)

where $\Sigma^{*}$ is the set of strings, so $\Agg$ must merge text and structures and not only average. Second, utility is an expectation over trajectories with sparse feedback, so it has no gradient in the discrete components and only noisy gradient estimates in the continuous ones. Third, a readable artifact can disclose a record verbatim, whereas a parameter update must first be inverted, so the leakage surface is wider. FAO is thus not merely federated model training, but collaborative optimization over policies, memories, tools, rewards, skills, and knowledge structures under privacy constraints.其中 Σ* 是字符串集合，因此 Agg 必须合并文本和结构，而不仅仅是取平均值。其次，效用是对具有稀疏反馈的轨迹的期望，因此在离散组件中没有梯度，在连续组件中只有带噪的梯度估计。第三，可读的工件可以逐字披露记录，而参数更新必须先反转，因此泄露面更广。因此，FAO 不仅仅是联邦模型训练，而是隐私约束下关于策略、记忆、工具、奖励、技能和知识结构的协作优化。

The objectives conflict, as quantified for federated learning of models [231, 232]: richer updates can raise $F_{1}$ , often at the expense of $F_{2}$ and $F_{3}$ . A protocol $\Phi$ with $F(\Phi)=(F_{1},F_{2},F_{3})$ is Pareto optimal if no other feasible protocol dominates it,这些目标存在冲突，正如联邦模型学习中所量化的那样 [231, 232]：更丰富的更新可以提高 F_1，但这通常以牺牲 F_2 和 F_3 为代价。如果不存在其他可行的协议支配协议 Φ，则其 F(Φ)=(F_1, F_2, F_3) 是帕累托最优的：

\nexists\,\Phi^{\prime}:\quad F_{i}(\Phi^{\prime})\geq F_{i}(\Phi)\ \ \forall i\in\{1,2,3\}\quad\text{and}\quad F(\Phi^{\prime})\neq F(\Phi).

(12)

FAO asks for methods that reach the Pareto-optimal set $\mathcal{Z}^{\star}$ and let a practitioner choose a point on the Pareto front $F(\mathcal{Z}^{\star})$ through one of two scalarized forms,FAO 要求寻找能够达到帕累托最优集 𝒵^⋆ 的方法，并让从业者通过以下两种标量化形式之一在帕累托前沿 F(𝒵^⋆) 上选择一个点：

		$\displaystyle\max_{\Phi}\ \frac{1}{K}\sum_{k=1}^{K}U_{k}\big(A_{k}^{+};\mathcal{Q}_{k}\big)-\lambda_{p}\,\frac{1}{K}\sum_{k=1}^{K}\mathcal{P}_{k}\big(M_{k}\big)-\lambda_{c}\,\mathcal{C}_{\mathrm{comm}}\big(\{\Delta_{k}\}_{k=1}^{K},\Phi\big),$		(13)
		$\displaystyle\max_{\Phi}\ \frac{1}{K}\sum_{k=1}^{K}U_{k}\big(A_{k}^{+};\mathcal{Q}_{k}\big)\quad\text{s.t.}\quad\mathcal{P}_{k}\big(M_{k}\big)\leq\bar{\epsilon}_{k}\ \ \forall k,\quad\mathcal{C}_{\mathrm{comm}}\big(\{\Delta_{k}\}_{k=1}^{K},\Phi\big)\leq B.$		(14)

The weights $\lambda_{p},\lambda_{c}>0$ express soft preferences, and the budgets $\bar{\epsilon}_{k}$ and $B$ express hard limits such as a regulatory requirement, which is how enterprises state their needs. For the exact maximizers $\mathcal{Z}_{\mathrm{w}}$ of Eq. (13) over all weights and $\mathcal{Z}_{\mathrm{c}}$ of Eq. (14) over all budgets, Proposition 1 in Appendix E and [124] give the inclusions权重 λ_p, λ_c > 0 表示软偏好，预算 ϵ_k 和 B 表示硬性限制（如监管要求），这是企业陈述其需求的方式。对于式 (13) 在所有权重上的精确最大化者 𝒵_w 和式 (14) 在所有预算上的 𝒵_c，附录 E 中的命题 1 和 [124] 给出了包含关系：

\mathcal{Z}_{\mathrm{w}}\subseteq\mathcal{Z}^{\star}\subseteq\mathcal{Z}_{\mathrm{c}},\qquad\Phi^{\star}\in\arg\max\eqref{eq:fao_constrained}\big|_{\bar{\epsilon}_{k}=\mathcal{P}_{k}(\Phi^{\star}),\ B=\mathcal{C}_{\mathrm{comm}}(\Phi^{\star})}\quad\forall\,\Phi^{\star}\in\mathcal{Z}^{\star}.

(15)

Positive weights yield only supported Pareto-optimal protocols, whose images lie on the convex hull of the front, and since the decision space contains text and rules and is not convex, only the constrained form is sure to reach every Pareto-optimal protocol.正权重仅产生受支持的帕累托最优协议，其图像位于前沿的凸包上；由于决策空间包含文本和规则且不是凸的，因此只有受约束的形式才能确保达到每一个帕累托最优协议。

A rational client joins the federation only if it gains from it, which adds a participation constraint on the collaboration gain of each client,理性的客户端仅在从中获益时才会加入联邦，这为每个客户端的协作收益增加了一个参与约束：

\mathrm{CG}_{k}=U_{k}\big(A_{k}^{+};\mathcal{Q}_{k}\big)-U_{k}\big(A_{k}^{\mathrm{loc}};\mathcal{Q}_{k}\big)\ \geq\ 0\qquad\forall k.

(16)

The gain follows from $F_{1}$ and a baseline, so it is not a fourth objective, and Section 5.3 gives a gate that enforces it with high probability, while Appendix D extends the formulation to inference cost and to task distributions that change over time.收益源于 F_1 和基准，因此它不是第四个目标。第 5.3 节给出了一个以高概率强制执行该约束的门控机制，而附录 D 将该公式扩展到了推理成本和随时间变化的任务分布。

3.4 Relation to Existing Paradigms3.4 与现有范式的关系

Several existing problems are special cases of Eq. (10),几个现有问题是式 (10) 的特例：

$\displaystyle\text{Federated Learning:}$	$\displaystyle\Delta_{k}=\big(\theta_{k}-\theta,\,\emptyset,\emptyset,\emptyset,\emptyset\big),\quad\theta^{+}=\theta+\textstyle\sum_{k}\frac{n_{k}}{n}\,\Delta_{k}^{\pi};$	(17)
$\displaystyle\text{Experience Pooling:}$	$\displaystyle\Delta_{k}=\mathcal{D}_{k},\quad\lambda_{p}=0;$
$\displaystyle\text{Agent Optimization:}$	$\displaystyle K=1,\quad\max_{A}\,U(A;\mathcal{Q});$
$\displaystyle\text{Component:}$	$\displaystyle\Delta_{k}=\big(\emptyset,\dots,\Delta_{k}^{c},\dots,\emptyset\big),\quad\lambda_{p}=\lambda_{c}=0.$

They cover federated averaging with $n=\sum_{k}n_{k}$ [122], multi-agent systems that pool raw messages or experience without a privacy objective [201, 100, 228, 174], single-agent optimization [88, 222], and recent methods that federate one component [35, 212, 200, 209] with privacy and communication fixed by design.它们涵盖了 n=∑_k n_k 的联邦平均 [122]、没有隐私目标且汇集原始消息或经验的多智能体系统 [201, 100, 228, 174]、单智能体优化 [88, 222]，以及近期将隐私和通信设计为固定的联邦化单个组件的方法 [35, 212, 200, 209]。

4. Taxonomy of Federated Agent Optimization4. 联邦智能体优化分类法

We classify FAO methods by the block of the typed update in Eq. (5) that they share (Table 1). We searched arXiv, the ACL Anthology, OpenReview, and the proceedings of the main machine learning venues up to 29 September 2026 for federated combined with agent, memory, tool, skill, reward model, prompt, or knowledge graph. Agent-level methods improve a component of a language agent across clients whose data stay local, and precursors federate the same kind of component outside an agent, such as the weights of a language model. The categories overlap (Appendix B).我们根据 FAO 方法在式 (5) 中共享的分类型更新块对其进行分类（表 1）。我们搜索了 arXiv、ACL Anthology、OpenReview 以及截至 2026 年 9 月 29 日的主要机器学习会议论文集，关键词包括“联邦”与“智能体”、“记忆”、“工具”、“技能”、“奖励模型”、“提示词”或“知识图谱”。智能体级方法在数据保持本地的客户端之间改进语言智能体的一个组件，而前驱方法则在智能体之外联邦化同类组件，例如语言模型的权重。这些类别存在重叠（附录 B）。

Category	Shared update	Main leakage channel	Agent-level methods	Precursors
Policy ( $\Delta^{\pi}$ )	adapter weights or gradients; soft or textual prompts	inversion of gradients; private examples kept in prompts	[186, 187, 185, 240], [35, 29]^∗	[229, 216, 170, 30, 51]
Memory ( $\Delta^{\mathcal{M}}$ )	insights, reasoning abstractions, workflows	private content quoted in text	[212, 109]^∗	[183], [2, 121]^∗
Tool use ( $\Delta^{\mathcal{T}}$ )	tool knowledge as text or typed fields; usage statistics	schemas of internal systems; arguments of calls	[200, 25]^∗	—
Reward ( $\Delta^{\mathcal{R}}$ )	parameters of evaluators or preference models; scores	labels and outcomes of individual cases	—	[199, 168], [90]^∗
Structured knowledge ( $\Delta^{\mathcal{O}}$ )	embeddings of entities and relations; rules; skill patches	entity sets and triples; business processes	[75, 209]^∗	[31, 138, 230]

Table 1: Taxonomy of FAO. Agent-level methods federate a component of a language agent, precursors the same kind of component outside an agent. ^∗Preprint.表 1：FAO 分类法。智能体级方法联邦化语言智能体的一个组件，前驱方法联邦化智能体之外的同类组件。*预印本。

Policy ( $\Delta^{\pi}$ ).策略 (Δ^π)。

With open weights, clients fine-tune adapters that the coordinator averages with weights $w_{k}$ , and with a closed backbone they share textual prompts $p_{k}$ that a language model merges, in both cases into one view for all clients,对于开放权重，客户端微调适配器，由协调器以权重 w_k 进行平均；对于封闭主干，它们共享文本提示词 p_k，由语言模型合并，两种情况下都形成供所有客户端使用的统一视图：

		$\displaystyle\text{weights:}\quad$	$\displaystyle\Delta_{k}^{\pi}$	$\displaystyle=\theta_{k}-\theta,\qquad$	$\displaystyle G^{\pi}$	$\displaystyle=\textstyle\sum_{k\in\mathcal{S}^{t}}w_{k}\,\Delta_{k}^{\pi},\qquad\theta_{k}^{+}=\theta+G^{\pi};$		(18)
		$\displaystyle\text{prompts:}\quad$	$\displaystyle\Delta_{k}^{\pi}$	$\displaystyle=p_{k},\qquad$	$\displaystyle G^{\pi}$	$\displaystyle=\mathrm{LLM}\big(p_{1},\dots,p_{K}\big),\qquad\theta_{k}^{+}=G^{\pi}.$		(18)

The first form is federated fine-tuning [229, 216, 203], with variants for heterogeneous adapter ranks [193, 38, 11], differential privacy [171], and personalization [211, 116, 59]. It has been applied to GUI agents [186, 187, 185], to agents tuned on their successful trajectories [35] or by reinforcement learning [29], and to workflows that span organizations [240]. The second form covers soft and black-box prompts [170, 113] and textual prompts [30], and it links FAO to the optimization of prompts and workflows by language models [208, 88, 222, 3]. Weights can be inverted to recover training text [246, 139, 163], against which secure aggregation and differential privacy are the standard defenses [18, 1], whereas prompts are small but readable by every receiver and may quote the private examples from which they were derived.第一种形式是联邦微调 [229, 216, 203]，具有针对异构适配器秩 [193, 38, 11]、差分隐私 [171] 和个性化 [211, 116, 59] 的变体。它已应用于 GUI 智能体 [186, 187, 185]、在成功轨迹上调优的智能体 [35] 或通过强化学习调优的智能体 [29]，以及跨组织的工作流 [240]。第二种形式涵盖软提示词和黑盒提示词 [170, 113] 以及文本提示词 [30]，并将 FAO 与语言模型对提示词和工作流的优化联系起来 [208, 88, 222, 3]。权重可以被反转以恢复训练文本 [246, 139, 163]，安全聚合和差分隐私是针对此的标准防御手段 [18, 1]；而提示词虽然较小，但可被每个接收者读取，并可能引用导出它们的私有示例。

Memory ( $\Delta^{\mathcal{M}}$ ).记忆 (Δ^ℳ)。

Raw trajectories stay local. Clients share items $z$ that an abstraction $\alpha$ distills from them, such as insights, workflows, and reasoning strategies [167, 237, 194, 134], the coordinator condenses the items of all clients, and each client retrieves the items $G^{\mathcal{M}}(x)$ that match its current task $x$ at inference time,原始轨迹保持本地。客户端共享由抽象 α 从中提取的项目 z，例如见解、工作流和推理策略 [167, 237, 194, 134]；协调器浓缩所有客户端的项目，每个客户端在推理时检索与其当前任务 x 匹配的项目 G^ℳ(x)：

\Delta_{k}^{\mathcal{M}}=\alpha(\mathcal{D}_{k})=\{z_{k,i}\}_{i},\qquad G^{\mathcal{M}}=\mathrm{Distill}\big(\textstyle\bigcup_{k}\Delta_{k}^{\mathcal{M}}\big),\qquad\mathcal{M}_{k}^{+}=\mathcal{M}_{k}\cup G^{\mathcal{M}}(x).

(19)

Systems with a trusted store share such items among agents [53, 228, 174, 157]. FoT and FedAgentKE avoid centralizing raw trajectories [212, 109], and federated retrieval-augmented generation avoids centralizing documents [24, 2, 238, 121]. Items are text, so $\Agg$ is performed by a language model, and the coordinator becomes an agent itself. Text can quote private content and stored memory can be extracted [223, 179, 128], so FoT audits its uploads by a reconstruction attack, and differentially private generation of text [175, 202, 204] would add a formal guarantee.具有可信存储的系统在智能体之间共享此类项目 [53, 228, 174, 157]。FoT 和 FedAgentKE 避免了原始轨迹的集中化 [212, 109]，联邦检索增强生成避免了文档的集中化 [24, 2, 238, 121]。项目是文本，因此 Agg 由语言模型执行，协调器本身也成为一个智能体。文本可能引用私有内容，存储的记忆也可能被提取 [223, 179, 128]，因此 FoT 通过重构攻击审计其上传内容，而差分隐私文本生成 [175, 202, 204] 将增加正式保证。

Tool use ( $\Delta^{\mathcal{T}}$ ).工具使用 (Δ^𝒯)。

Clients in the horizontal setting use tools of the same kind, and standard interfaces make their descriptions comparable [8]. For a tool $v$ and an intent $c$ , a client shares its documentation $\mathrm{doc}_{k}(v)$ and its numbers of calls $N_{k}$ and successes $N_{k}^{+}$ , from which the coordinator derives a merged documentation and a success rate $\hat{q}$ ,水平设置下的客户端使用同类工具，标准接口使其描述具有可比性 [8]。对于工具 v 和意图 c，客户端共享其文档 doc_k(v) 以及调用次数 N_k 和成功次数 N_k^+，协调器据此导出合并后的文档和成功率 q^：

\Delta_{k}^{\mathcal{T}}=\big(\mathrm{doc}_{k},N_{k},N_{k}^{+}\big),\qquad\mathrm{doc}(v)=\mathrm{Merge}\big(\{\mathrm{doc}_{k}(v)\}_{k}\big),\qquad\hat{q}(v,c)=\frac{\sum_{k}N_{k}^{+}(v,c)}{\sum_{k}N_{k}(v,c)}.

(20)

Only the sums of the counts are needed, so secure aggregation protects them [18]. Descriptions decide how well a tool is used, and they can be refined from experience [165, 150, 220, 152]. FICAL concatenates the textual tool knowledge of the clients [200], and typed artifacts merge it field by field across frozen models of different vendors [25]. Schemas disclose internal systems, and a shared description is an instruction to every agent that reads it, so $\Agg$ must also resist malicious contributions [74].仅需计数之和，因此安全聚合可保护它们 [18]。描述决定了工具的使用效果，并可根据经验进行优化 [165, 150, 220, 152]。FICAL 连接了客户端的文本工具知识 [200]，分类型工件将其跨不同供应商的冻结模型逐字段合并 [25]。模式会披露内部系统，共享描述是对阅读它的每个智能体的指令，因此 Agg 还必须抵御恶意贡献 [74]。

Reward ( $\Delta^{\mathcal{R}}$ ).奖励 (Δ^ℛ)。

Clients observe real outcomes but few of them, and the coordinator can run a strong judge, but judges are biased and unreliable on trajectories [243, 115]. A shared evaluator $g_{\phi}$ and a local head $h_{\psi}$ , trained jointly with a loss $\ell$ , combine both sides, and only the shared part $g_{\phi}$ leaves the client,客户端观察到真实结果但数量很少，协调器可以运行一个强大的评判器，但评判器在轨迹上存在偏差且不可靠 [243, 115]。共享评估器 g_ϕ 和本地头 h_ψ（通过损失 ℓ 联合训练）结合了双方，只有共享部分 g_ϕ 离开客户端：

(\phi_{k},\psi_{k})=\arg\min_{\phi,\psi}\sum_{i=1}^{n_{k}}\ell\big(h_{\psi}(g_{\phi}(\tau_{k,i})),\,u_{k}(\tau_{k,i})\big),\quad\Delta_{k}^{\mathcal{R}}=\phi_{k}-\phi,\quad\mathcal{R}_{k}^{+}=h_{\psi_{k}}\circ g_{\phi+G^{\mathcal{R}}}.

(21)

Existing methods federate preference or reward models for aligning language models [199, 168, 90] and handle heterogeneous $u_{k}$ by grouping similar clients [199] or by an initialization that adapts quickly [90], whereas FedRLHF keeps rewards local and federates policies [51] and FedEval-LLM uses client models as referees [70]. To our knowledge, none federates an evaluator of agent trajectories, for which process reward models [110, 23] and reward functions written by language models [94, 119, 206] are building blocks. A cheaper form shares only the scores of candidate rewards $r^{(j)}$ from the coordinator,现有方法联邦化用于对齐语言模型的偏好或奖励模型 [199, 168, 90]，并通过对相似客户端分组 [199] 或通过快速适应的初始化 [90] 来处理异构的 u_k；而 FedRLHF 保持奖励本地化并联邦化策略 [51]，FedEval-LLM 使用客户端模型作为裁判 [70]。据我们所知，尚无方法联邦化智能体轨迹的评估器，过程奖励模型 [110, 23] 和由语言模型编写的奖励函数 [94, 119, 206] 是其构建块。一种更廉价的形式仅共享来自协调器的候选奖励 r(j) 的分数：

\Delta_{k}^{\mathcal{R}}=\big(\kappa_{k}^{(j)}\big)_{j=1}^{J},\qquad\kappa_{k}^{(j)}=\operatorname*{corr}_{i\leq n_{k}}\big(r^{(j)}(\tau_{k,i}),\,u_{k}(\tau_{k,i})\big),\qquad G^{\mathcal{R}}=\arg\max_{r^{(j)}}\,\textstyle\sum_{k}\kappa_{k}^{(j)}.

(22)

Only $J$ scores leave each client, so leakage and payload stay small.每个客户端仅离开 J 个分数，因此泄露和有效载荷保持较小。

Structured knowledge and skills ( $\Delta^{\mathcal{O}}$ ).结构化知识和技能 (Δ^𝒪)。

For an entity $e$ with local embedding $\mathbf{e}_{k}$ , known to the clients $\mathcal{K}_{e}$ , and for a skill library $\mathcal{L}_{k}$ with local patch $\delta_{k}$ ,对于具有本地嵌入 e_k 的实体 e（为客户端 𝒦_e 所知），以及具有本地补丁 δ_k 的技能库 ℒ_k：

G^{\mathcal{O}}(e)=\frac{1}{|\mathcal{K}_{e}|}\sum_{k\in\mathcal{K}_{e}}\mathbf{e}_{k},\qquad\mathcal{L}_{k}^{+}=\mathrm{Apply}\Big(\mathcal{L}_{k},\ G_{k}^{\mathcal{O}}\Big),\qquad G_{k}^{\mathcal{O}}=\mathrm{Merge}_{k}\big(\{\delta_{j}\}_{j=1}^{K}\big).

(23)

FedE averages entity embeddings [31]. They allow triples to be reconstructed, so FedR shares only relation embeddings under secure aggregation [230], and FKGE and FedLU add differential privacy and personalization [138, 247]. All assume aligned entities, which ontology matching provides, now also with language models [48, 10, 148]. FedWorld federates rules on environment dynamics and labels each rule as shared, specific to a cluster, private, or unresolved [75]. FederatedSkill exchanges patches to skill libraries, which an agent at the server merges into a personalized view $G_{k}^{\mathcal{O}}$ for each client [209]. Embeddings reveal which entities a client knows, and skills how it conducts its business.FedE 对实体嵌入进行平均 [31]。它们允许重构三元组，因此 FedR 在安全聚合下仅共享关系嵌入 [230]，FKGE 和 FedLU 增加了差分隐私和个性化 [138, 247]。所有方法都假设实体已对齐，本体匹配（现在也使用语言模型 [48, 10, 148]）提供了这一点。FedWorld 联邦化环境动态规则，并将每条规则标记为共享、特定于集群、私有或未解决 [75]。FederatedSkill 交换技能库补丁，服务器上的智能体将其合并为每个客户端的个性化视图 G_k^𝒪 [209]。嵌入揭示了客户端知道哪些实体，技能揭示了它如何开展业务。

Comparison.比较。

The reported payloads differ by roughly three orders of magnitude,报告的有效载荷差异约为三个数量级：

|\Delta^{\pi}|\approx 76~\text{MB},\qquad|\Delta^{\mathcal{T}}|\approx 0.08\text{ to }20~\text{MB},\qquad|\Delta^{\mathcal{O}}|\approx 0.1\text{ to }0.3~\text{MB},

(24)

for adapters of rank 8 per round [35], tool knowledge per run or per round [200, 25], and skill patches per run [209], and parameter updates can also be compressed to kilobytes [151]. The methods that exchange text or rules [200, 212, 109, 209, 75] keep raw trajectories local but apply no formal privacy mechanism, and two of them audit their uploads empirically [212, 209]. To our knowledge, no method trades the three objectives off across components, which is what Eq. (10) asks for.针对每轮秩为 8 的适配器 [35]、每次运行或每轮的工具知识 [200, 25] 以及每次运行的技能补丁 [209]，参数更新也可以压缩到千字节 [151]。交换文本或规则的方法 [200, 212, 109, 209, 75] 将原始轨迹保持在本地，但未应用正式的隐私机制，其中两种方法对上传内容进行了实证审计 [212, 209]。据我们所知，没有方法跨组件权衡这三个目标，而这正是式 (10) 所要求的。

5. How Agents Mutually Improve Each Other5. 智能体如何相互改进

Section 4 classified FAO by the component that is optimized. This section follows the knowledge, from the private experience of one agent to the gain of another, with a focus on semantic updates such as text, rules, and skills.第 4 节按优化的组件对 FAO 进行了分类。本节沿知识路径，从一个智能体的私有经验到另一个智能体的收益，重点关注文本、规则和技能等语义更新。

5.1 The Improvement Loop5.1 改进循环

Every update maps local evidence, such as trajectories, records, or tool interactions, to an artifact and protects it, $\Update=\rho\circ\alpha$ , where $\alpha$ is local training for parametric updates and an abstraction $\alpha_{\ell}$ at a level $\ell$ for semantic ones, which carry knowledge from a sender $j$ to a receiver $k$ in four stages under a protection rule,每次更新将本地证据（如轨迹、记录或工具交互）映射到工件并加以保护，Update = ρ ∘ α，其中 α 是参数更新的本地训练，α_ℓ 是语义更新在级别 ℓ 的抽象，它们在保护规则下分四个阶段将知识从发送者 j 传递给接收者 k：

		$\displaystyle\mathcal{D}_{j}\ \xrightarrow{\ \alpha_{\ell}\ }\ z\ \xrightarrow{\ \rho\ }\ \tilde{z}\ \xrightarrow{\ \Agg\ }\ G_{k}\ \xrightarrow{\ \Adapt\ }\ A_{k}^{+},$		(25)
		$\displaystyle\rho(z)=\mathrm{Sanitize}(z)\ \ \text{if }\big\|\{e\in\mathcal{D}_{j}:e\models z\}\big\|\geq n_{\min},\qquad\rho(z)=\emptyset\ \ \text{otherwise}.$		(26)

Here the predicate $e\models z$ , evaluated by the sender, states that the evidence item $e$ exhibits the pattern $z$ , $n_{\min}$ is a support threshold, and $\mathrm{Sanitize}$ removes identifiers or adds calibrated noise. Abstraction is a central design dimension of FAO, because knowledge travels as text, rules, and skills, and not only as parameters or as the logits of federated distillation [99, 52]. The levels $\boldsymbol{\ell}=(\ell_{1},\dots,\ell_{K})$ trade the detail of the artifacts against their size, leakage, and reach, and as part of $\Phi$ they are chosen jointly by Eq. (13),此处由发送者评估的谓词 e ⊧ z 表示证据项 e 展示了模式 z，n_min 是支持阈值，Sanitize 移除标识符或添加校准噪声。抽象是 FAO 的核心设计维度，因为知识不仅作为参数或联邦蒸馏的对数 [99, 52] 传播，还作为文本、规则和技能传播。级别 ℓ = (ℓ_1, …, ℓ_K) 在工件的细节与其大小、泄露和覆盖范围之间进行权衡，并作为 Φ 的一部分由式 (13) 联合选择：

\boldsymbol{\ell}^{\star}\in\arg\max_{\boldsymbol{\ell}}\ F_{1}(\boldsymbol{\ell})-\frac{\lambda_{p}}{K}\sum_{k=1}^{K}\mathcal{P}_{k}(\ell_{k})-\lambda_{c}\,\mathcal{C}_{\mathrm{comm}}(\boldsymbol{\ell}).

(27)

A selfish client would instead maximize its own utility minus its own costs and, since its upload mainly helps others, upload as little as possible, the free-riding that incentives counter (Section 5.3). Higher abstraction should shrink $z$ , lower leakage, and widen reach, though not always: an abstract business rule can itself be a trade secret.自私的客户端会最大化其自身效用减去自身成本，由于其上传内容主要帮助他人，因此会尽可能少地上传，这是激励机制所要对抗的搭便车行为（第 5.3 节）。更高的抽象度应该会缩小 z、降低泄露并扩大覆盖范围，尽管并非总是如此：抽象的业务规则本身可能就是商业机密。

5.2 Five Types of Shared Knowledge5.2 五种共享知识类型

Table 2 lists five types of shared knowledge with a running example, in which banks verify customer identity with agents and cannot share records. The types differ in function rather than in degree of abstraction, and they interact: aligned ontologies make experience, skills, and rules easier to ground (Eq. 29), and the reward type supplies the evaluator that selects what the sender distills. An artifact of any type may enter the memory, the prompt, or the evaluator of the receiving client (Appendix C).表 2 列出了五种共享知识类型及其运行示例，其中银行通过智能体验证客户身份且无法共享记录。这些类型在功能上而非抽象程度上有所差异，且它们相互作用：对齐的本体使经验、技能和规则更容易落地（式 29），奖励类型提供了选择发送者蒸馏内容的评估器。任何类型的工件都可能进入接收客户端的记忆、提示词或评估器中（附录 C）。

Type	Shared artifact $z$	Example in identity verification	Federated methods
Experience	pattern distilled from successes and failures	normalize the transliteration of names before the registry lookup	[212, 109]
Skill	parameterized procedure	recognize the license, query the registry, compare the legal representative, escalate on mismatch	[209]
Rule	effect of an action in the environment	three failed one-time codes lock the account for 30 minutes	[75]
Reward	evaluator or rubric	a dialogue is non-compliant if the agent reveals which field failed the check	[199, 90]^†
Ontology	alignment of concepts	beneficial owner $\equiv$ ultimate controlling person	[31, 138]^†

Table 2: Five types of shared knowledge, which differ in function, with a running example. ^†Precursor that federates the component outside an agent (Table 1).表 2：五种功能不同的共享知识类型及其运行示例。†在智能体之外联邦化该组件的前驱方法（表 1）。

5.3 When Is the Improvement Mutual?5.3 改进何时是相互的？

Let $A_{k}^{+\setminus j}$ be the agent that client $k$ obtains when the updates of client $j$ are withheld. The contribution of $j$ to $k$ and the condition for mutual improvement are令 A_k^{+\j} 为客户端 k 在扣除客户端 j 的更新后获得的智能体。j 对 k 的贡献以及相互改进的条件为：

I_{j\to k}=U_{k}\big(A_{k}^{+};\mathcal{Q}_{k}\big)-U_{k}\big(A_{k}^{+\setminus j};\mathcal{Q}_{k}\big),\qquad I_{j\to k}>0\ \ \text{and}\ \ I_{k\to j}>0.

(28)

Averaged over subsets of clients, such marginal contributions give the Shapley value, which is used to evaluate contributions in federated learning [56, 184]. Three conditions favor the transfer of an item $z$ from $j$ to $k$ , with a divergence $\mathrm{div}$ of task distributions, a novelty threshold $\nu$ for known items $y\in\mathcal{M}_{k}\cup\mathcal{O}_{k}$ , an alignment $\mu_{k}$ of shared to local concepts, and the concepts $\mathrm{con}(z)$ and tools $\mathrm{tool}(z)$ that $z$ uses,在客户端子集上平均后，此类边际贡献给出了 Shapley 值，该值用于评估联邦学习中的贡献 [56, 184]。三个条件有利于项 z 从 j 迁移到 k：任务分布的散度 div、已知项 y ∈ ℳ_k ∪ 𝒪_k 的新颖性阈值 ν、共享概念与本地概念的对齐 μ_k，以及 z 使用的概念 con(z) 和工具 tool(z)：

\underbrace{\mathrm{div}(\mathcal{Q}_{j},\mathcal{Q}_{k})\leq\mathrm{div}_{\max}}_{\text{relatedness}},\quad\underbrace{\max\nolimits_{y}\mathrm{sim}(z,y)<\nu}_{\text{novelty}},\quad\underbrace{\mathrm{con}(z)\subseteq\operatorname{dom}(\mu_{k}),\ \ \mathrm{tool}(z)\subseteq\mathcal{T}_{k}}_{\text{groundability}}.

(29)

They are heuristics, not necessary conditions: distant tasks can share a strategy, a known item can add robustness, and an item can bring a new concept or tool. Heterogeneity is thus both an obstacle to transfer and a source of its value. Clustered aggregation addresses relatedness [57, 164], and agents trained by federated reinforcement learning tolerate differences in tasks but not in environment dynamics [29].这些是启发式方法，而非必要条件：遥远的任务可以共享策略，已知项可以增加鲁棒性，项可以带来新概念或工具。因此，异构性既是迁移的障碍，也是其价值的来源。聚类聚合解决了相关性问题 [57, 164]，通过联邦强化学习训练的智能体可以容忍任务差异，但不能容忍环境动态差异 [29]。

Two mechanisms address these risks. A gate against negative transfer [192] adds the items $z_{1},\dots,z_{m}$ one at a time to $A_{k}^{(0)}=A_{k}^{\mathrm{loc}}$ , where $A\oplus z$ integrates $z$ into $A$ , and accepts an item only if its estimated gain exceeds twice an error bound $e_{k}$ ,两种机制解决了这些风险。针对负迁移的门控 [192] 将项 z_1, …, z_m 一个接一个地添加到 A_k^{(0)} = A_k^loc 中，其中 A ⊕ z 将 z 集成到 A 中，并且仅在估计收益超过两倍误差界 e_k 时才接受该项：

A_{k}^{(i)}=\begin{cases}A_{k}^{(i-1)}\oplus z_{i}&\text{if }\hat{U}_{k}\big(A_{k}^{(i-1)}\oplus z_{i}\big)-\hat{U}_{k}\big(A_{k}^{(i-1)}\big)\geq 2e_{k},\\ A_{k}^{(i-1)}&\text{otherwise},\end{cases}\qquad i=1,\dots,m.

(30)

With $A_{k}^{+}=A_{k}^{(m)}$ and $\hat{U}_{k}$ the mean of $u_{k}$ on $n_{k}^{\mathrm{val}}$ held-out tasks from $\mathcal{Q}_{k}$ , Hoeffding’s inequality and a union bound over the $2^{m}$ reachable agents give其中 A_k^+ = A_k^{(m)}，U_k^hat 是 u_k 在来自 𝒬_k 的 n_k^val 个留出任务上的平均值，Hoeffding 不等式和对 2^m 个可达智能体的联合界给出了：

e_{k}=\sqrt{\frac{(m+1)\ln 2+\ln(1/\eta)}{2\,n_{k}^{\mathrm{val}}}}\quad\Longrightarrow\quad\Pr\big[\mathrm{CG}_{k}\geq 0\big]\ \geq\ 1-\eta,

(31)

so by Proposition 2 the gate guarantees individual rationality, that is, the participation constraint (16), at a margin that grows with $m$ and calls for filtering or batching the items. Pairwise mutual improvement in the sense of Eq. (28) is stronger and not implied by Proposition 2, because the items of different clients interact. Incentives that tie the benefit of a client to its contribution $I_{j\to k}$ encourage it, since a client that only adopts knowledge gains without bearing the privacy cost [16, 44, 224, 92]. Related safeguards: FedWorld checks the scope of each rule [75], Agent KB filters retrieved knowledge by a disagreement gate [174], and FederatedSkill builds a library per client [209]. Each local agent thus remains private and personalized, while the federation lets it benefit from the experience that other agents accumulate in environments it never sees.因此根据命题 2，该门控保证了个人理性，即参与约束 (16)，其裕度随 m 增长，这要求对项进行过滤或批处理。式 (28) 意义上的成对相互改进更强，且命题 2 未隐含它，因为不同客户端的项会相互作用。将客户端利益与其贡献 I_{j→k} 挂钩的激励机制鼓励了这一点，因为只采用知识而不承担隐私成本的客户端会获益 [16, 44, 224, 92]。相关保障：FedWorld 检查每条规则的范围 [75]，Agent KB 通过分歧门控过滤检索到的知识 [174]，FederatedSkill 为每个客户端构建一个库 [209]。因此，每个本地智能体保持私有和个性化，而联邦化使其能够从其他智能体在它从未见过的环境中积累的经验中获益。

6. Key Challenges6. 关键挑战

The mechanisms above illustrate how local experience can be transformed into transferable capabilities and ultimately improve other agents. However, they also expose three fundamental challenges that distinguish FAO from conventional federated learning. First, performance, privacy, and communication are inherently coupled rather than independently optimizable objectives. Second, heterogeneous participants may contribute and benefit unevenly, making fairness central to sustained collaboration. Third, the gains of federation are difficult to verify because agent outcomes are long-horizon, environment-dependent, and not yet supported by dedicated evaluation protocols. These challenges extend the statistical and system heterogeneity already studied in federated learning [86, 96] from model optimization to the broader problem of collaborative agent improvement.上述机制说明了本地经验如何转化为可迁移的能力并最终改进其他智能体。然而，它们也暴露了三个区分 FAO 与传统联邦学习的基本挑战。首先，性能、隐私和通信是内在耦合的，而非可独立优化的目标。其次，异构参与者的贡献和收益可能不均，使得公平性成为持续协作的核心。第三，联邦化的收益难以验证，因为智能体的结果是长周期的、依赖于环境的，且尚未得到专门评估协议的支持。这些挑战将联邦学习中已经研究过的统计和系统异构性 [86, 96] 从模型优化扩展到了协作智能体改进这一更广泛的问题。

6.1 Performance, Privacy and Communication Trade-offs6.1 性能、隐私和通信的权衡

The three objectives of FAO—task performance, privacy protection, and communication efficiency—form a tightly coupled trade-off. Improving performance generally requires sharing information that preserves sufficient detail about local experience, yet richer updates can increase both privacy exposure and communication overhead. Data locality alone does not eliminate this risk: gradients and model updates can reveal information about local training examples [54, 129], while memories, skill descriptions, tool-use knowledge, and structured artifacts may disclose sensitive content more directly. Stronger protection, such as differential privacy, reduces disclosure but can also degrade the information available for optimization [1]. Conversely, aggressive compression can substantially reduce communication traffic [112], but may remove information that is important for capability transfer. Secure aggregation protects individual updates from direct inspection [18], yet this can further constrain validation of what is being shared. Client heterogeneity additionally changes the utility of the same update across participants and can induce objective inconsistency in global optimization [181]. The central challenge is therefore not to minimize any single cost in isolation, but to identify operating points at which useful capability transfer remains possible under explicit privacy and communication budgets. This makes the Pareto trade-off defined in Section 3 a central systems question for practical FAO.FAO 的三个目标——任务性能、隐私保护和通信效率——形成了紧密耦合的权衡。提高性能通常需要共享保留足够本地经验细节的信息，然而更丰富的更新可能会增加隐私暴露和通信开销。仅靠数据本地化并不能消除这种风险：梯度和模型更新可能揭示有关本地训练示例的信息 [54, 129]，而记忆、技能描述、工具使用知识和结构化工件可能更直接地披露敏感内容。更强的保护（如差分隐私）减少了披露，但也可能降低可用于优化的信息质量 [1]。相反，激进的压缩可以大幅减少通信流量 [112]，但可能会移除对能力迁移重要的信息。安全聚合保护单个更新免受直接检查 [18]，但这可能会进一步限制对共享内容的验证。客户端异构性还会改变同一更新在不同参与者之间的效用，并可能导致全局优化中的目标不一致 [181]。因此，核心挑战不在于孤立地最小化任何单一成本，而在于确定在明确的隐私和通信预算下仍能实现有用能力迁移的操作点。这使得第 3 节定义的帕累托权衡成为实际 FAO 的核心系统问题。

6.2 How to Ensure Fairness in Federated Agent Systems?6.2 如何确保联邦智能体系统中的公平性？

A second challenge is ensuring that the benefits and burdens of federation are distributed fairly across heterogeneous participants. Improvements in federation-wide utility may coexist with substantially different outcomes for individual clients [108, 127]. Some agents may contribute highly valuable experience but receive limited benefit in return, whereas others may benefit primarily from knowledge produced elsewhere. At the same time, globally shared capabilities can induce negative transfer when local objectives, task distributions, tools, or operating constraints differ. Personalized federated learning provides mechanisms for retaining local specialization under shared optimization [107, 172], but FAO requires a broader notion of fairness that also accounts for non-parametric artifacts such as memories, skills, rewards, and structured knowledge. Contribution estimation is itself difficult because task outcomes result from long sequences of decisions and feedback may be sparse or delayed [9]; attributing recipient-side gains to particular shared contributions introduces an additional layer of credit assignment [68]. The collaboration gain and pairwise contribution defined in Section 5 therefore provide important starting points, but a complete fairness framework must jointly consider what each participant contributes, what benefit it receives, and what privacy, computation, and communication costs it bears. Trustworthy participation ultimately requires that clients retain the ability to personalize, selectively adopt, or reject shared capabilities whose local value does not justify their associated cost or risk.第二个挑战是确保联邦化的利益和负担在异构参与者之间公平分配。联邦范围内的效用提升可能伴随着单个客户端截然不同的结果 [108, 127]。一些智能体可能贡献了非常有价值的经验，但获得的收益有限，而另一些智能体可能主要受益于其他地方产生的知识。同时，当本地目标、任务分布、工具或操作约束不同时，全局共享的能力可能导致负迁移。个性化联邦学习提供了在共享优化下保留本地专业化的机制 [107, 172]，但 FAO 需要一种更广泛的公平性概念，该概念还要考虑非参数化工件，如记忆、技能、奖励和结构化知识。贡献估计本身就很困难，因为任务结果源于长序列的决策，且反馈可能是稀疏或延迟的 [9]；将接收方的收益归因于特定的共享贡献引入了额外的信用分配层 [68]。因此，第 5 节定义的协作收益和成对贡献提供了重要的起点，但一个完整的公平性框架必须联合考虑每个参与者贡献了什么、获得了什么收益，以及承担了什么隐私、计算和通信成本。可信参与最终要求客户端保留个性化、选择性采用或拒绝那些本地价值无法证明其相关成本或风险的共享能力的权利。

6.3 How to Evaluate Federated Agent Systems?6.3 如何评估联邦智能体系统？

A third challenge is establishing credible evidence that federated collaboration actually improves agent capability. Agent performance is inherently difficult to verify because success depends on interactions among reasoning, memory retrieval, tool execution, and environmental conditions. Tool-use research shows that apparently similar tools and interfaces can require substantially different execution behavior, while reasoning and environment-facing actions are tightly coupled in interactive agents [214]. Consequently, a shared artifact that appears semantically valid may still fail after transfer to a recipient environment. Verification becomes even harder over time: continual optimization can introduce stale capabilities, distribution drift, and interference with previously acquired knowledge [218, 233]. Erroneous or adversarial contributions further complicate the distinction between genuine capability improvement and harmful shared updates [217]. Existing benchmarks evaluate general agent competence, web and desktop interaction, or tool–user workflows [114, 244, 205, 213], while federated benchmarks such as FedScale primarily characterize model and system heterogeneity [95]. FAO requires evaluation protocols that combine these perspectives: controlled client heterogeneity, explicit information-sharing boundaries, repeated collaborative optimization, and credible local baselines. Beyond federation-wide task utility, evaluation should report privacy leakage, communication and system costs, client-level collaboration gains, contribution fairness, negative-transfer rates, and robustness under task or participant drift. Without such benchmarks and metrics, it remains difficult to determine when federation truly creates transferable intelligence rather than merely improving an aggregate score.第三个挑战是建立可信的证据，证明联邦协作确实提升了智能体能力。智能体的表现本质上难以验证，因为其成功取决于推理、记忆检索、工具执行和环境条件之间的相互作用。工具使用研究表明，表面上相似的工具和接口可能需要截然不同的执行行为，而推理和面向环境的操作在交互式智能体中是紧密耦合的 [214]。因此，一个语义上看起来有效的共享工件在转移到接收环境后仍可能失效。随着时间的推移，验证变得更加困难：持续优化可能会引入过时的能力、分布偏移，并干扰先前获得的知识 [218, 233]。错误或对抗性的贡献进一步增加了区分真正的能力提升与有害的共享更新的难度 [217]。现有的基准测试评估的是通用智能体能力、网页和桌面交互或工具-用户工作流 [114, 244, 205, 213]，而 FedScale 等联邦基准测试主要刻画的是模型和系统的异构性 [95]。FAO 需要结合这些视角的评估协议：受控的客户端异构性、明确的信息共享边界、重复的协作优化以及可信的本地基准。除了联邦范围内的任务效用外，评估还应报告隐私泄露、通信和系统成本、客户端层面的协作收益、贡献公平性、负迁移率以及在任务或参与者发生漂移时的稳健性。如果没有这样的基准和指标，就很难确定联邦究竟是创造了可转移的智能，还是仅仅提高了汇总分数。

7. Future Works7. 未来工作

A central direction for future FAO research is to build trustworthy federated agent systems, where trustworthiness is treated as an end-to-end property of collaborative optimization rather than a single notion of model accuracy or privacy. We highlight several important research questions.未来 FAO 研究的一个核心方向是构建可信的联邦智能体系统，其中可信度被视为协作优化的端到端属性，而非模型准确性或隐私的单一概念。我们强调了几个重要的研究问题。

❶How can transferred capabilities remain reliable and executable across heterogeneous agent environments? A capability that is effective for one agent may rely on specific tools, interfaces, permissions, task distributions, or workflow contexts that do not hold for another agent. Prior work on grounded planning, tool use, and multi-agent workflows highlights the importance of environment-aware execution and capability composition [77, 67, 104, 73]. Future FAO systems should therefore explicitly model the applicability conditions, dependencies, and execution constraints of transferred capabilities, rather than assuming that capabilities remain valid after cross-agent transfer.❶ 转移的能力如何在异构智能体环境中保持可靠性和可执行性？对一个智能体有效的能力可能依赖于特定的工具、接口、权限、任务分布或工作流上下文，而这些条件在另一个智能体中可能并不成立。先前关于扎根规划、工具使用和多智能体工作流的研究强调了环境感知执行和能力组合的重要性 [77, 67, 104, 73]。因此，未来的 FAO 系统应明确建模转移能力的适用条件、依赖关系和执行约束，而不是假设能力在跨智能体转移后依然有效。

❷How can federated agent systems protect privacy while remaining secure against unreliable or malicious contributions? Keeping raw data local does not eliminate leakage from model parameters, memories, textual abstractions, tool knowledge, or structured representations. Differential privacy and secure aggregation provide useful foundations for limiting such leakage [1, 18], while robust aggregation is necessary to tolerate faulty or adversarial participants [15, 21]. Future research should jointly consider confidentiality, update integrity, and the tension between protecting shared information and validating its quality.❷ 联邦智能体系统如何在保护隐私的同时，防范不可靠或恶意贡献？将原始数据保留在本地并不能消除模型参数、记忆、文本抽象、工具知识或结构化表示带来的泄露。差分隐私和安全聚合为限制此类泄露提供了有用的基础 [1, 18]，而稳健的聚合对于容忍故障或对抗性参与者是必要的 [15, 21]。未来的研究应综合考虑机密性、更新完整性，以及保护共享信息与验证其质量之间的张力。

❸How can fair contribution and benefit allocation be achieved among heterogeneous agents? Different agents may contribute unequal amounts of useful experience, receive different levels of benefit, or even suffer negative transfer. Personalized and fairness-aware federated learning provide useful foundations for preserving local specialization and participant-level utility [49, 82, 156, 111]. Future FAO research should further quantify contribution, collaboration gain, and participation cost, and ensure that agents receive benefits commensurate with the information, privacy, and resources they contribute.❸ 如何在异构智能体之间实现公平的贡献和利益分配？不同的智能体可能贡献不等量的有用经验，获得不同程度的收益，甚至遭受负迁移。个性化和公平感知的联邦学习为保留本地专业化和参与者层面的效用提供了有用的基础 [49, 82, 156, 111]。未来的 FAO 研究应进一步量化贡献、协作收益和参与成本，并确保智能体获得的利益与其贡献的信息、隐私和资源相称。

❹How should federated agent systems be evaluated? Reliable evaluation should verify whether collaboration genuinely improves local agents compared with credible local baselines, rather than relying only on shared evaluators or aggregate performance. This is particularly challenging when feedback is sparse, trajectories are long, and environments are stochastic [39, 133, 154, 120]. Existing agent and federated benchmarks provide useful starting points [114, 244, 205, 213, 95], but FAO additionally requires explicit information-sharing boundaries, heterogeneous participants, repeated collaboration, and longitudinal evaluation of recipient-level gains, negative transfer, and robustness under drift.❹ 联邦智能体系统应如何评估？可靠的评估应验证协作是否确实改善了本地智能体（相比于可信的本地基准），而不是仅仅依赖共享评估器或汇总性能。当反馈稀疏、轨迹较长且环境是随机时，这一点尤其具有挑战性 [39, 133, 154, 120]。现有的智能体和联邦基准测试提供了有用的起点 [114, 244, 205, 213, 95]，但 FAO 还需要明确的信息共享边界、异构参与者、重复协作，以及对接收者层面的收益、负迁移和漂移下稳健性的纵向评估。

❺How should the full lifecycle cost of federated agent collaboration be measured and optimized? Communication volume alone is insufficient to characterize the cost of FAO. Collaboration may also incur substantial costs for local adaptation, privacy protection, validation, inference, tool execution, storage, synchronization, and continual maintenance, especially under heterogeneous computation and model capacities [4, 123, 43]. Future work should therefore develop lifecycle-level cost models that jointly account for the resources required to produce, protect, aggregate, validate, deploy, and maintain shared capabilities.❺ 如何衡量和优化联邦智能体协作的全生命周期成本？仅靠通信量不足以刻画 FAO 的成本。协作还可能产生大量的本地适配、隐私保护、验证、推理、工具执行、存储、同步和持续维护成本，特别是在计算能力和模型能力异构的情况下 [4, 123, 43]。因此，未来的工作应开发生命周期层面的成本模型，综合核算生产、保护、聚合、验证、部署和维护共享能力所需的资源。

Taken together, these questions suggest that trustworthy FAO should jointly consider execution reliability, privacy and security, fairness, credible evaluation, and lifecycle cost, providing a principled basis for collaborative learning among autonomous agents across organizations.综上所述，这些问题表明，可信的 FAO 应综合考虑执行可靠性、隐私与安全、公平性、可信评估和生命周期成本，从而为跨组织的自主智能体之间的协作学习提供原则性基础。

8. Conclusion8. 结论

This paper formulates Federated Agent Optimization (FAO) as a general framework for collaborative improvement across policies, memories, tool use, rewards, and structured knowledge under distributed data and local constraints. By extending federated optimization beyond model parameters, we establish a multi-objective problem formulation, a taxonomy of federated agent components, and an improvement framework that connects local experience to transferable capabilities and recipient-side gains. Our analysis highlights three central challenges—balancing performance, privacy, and communication cost, ensuring fair contribution and benefit allocation, and reliably verifying the gains of federation—which together motivate the broader goal of building trustworthy FAO systems with reliable execution, privacy and security protection, credible evaluation, and principled cost accounting. We envision FAO as a foundation for scalable, trustworthy, and continuously evolving enterprise AI systems that benefit from collective experience while preserving local specialization and organizational control.本文将联邦智能体优化（FAO）表述为一个通用框架，旨在分布式数据和本地约束下，跨策略、记忆、工具使用、奖励和结构化知识实现协作改进。通过将联邦优化扩展到模型参数之外，我们建立了多目标问题表述、联邦智能体组件分类法，以及一个将本地经验与可转移能力及接收端收益相连接的改进框架。我们的分析强调了三个核心挑战——平衡性能、隐私与通信成本，确保公平的贡献与利益分配，以及可靠地验证联邦收益——这些挑战共同推动了构建可信 FAO 系统的更广泛目标，即实现可靠的执行、隐私与安全保护、可信的评估以及原则性的成本核算。我们将 FAO 设想为可扩展、可信且持续演进的企业级 AI 系统的基础，使其既能从集体经验中获益，又能保持本地专业化和组织控制。

References参考文献

[1] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang (2016) Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318. Cited by: §B.1, §3.3, §4, §6.1, §7. [1] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang (2016) Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318. Cited by: §B.1, §3.3, §4, §6.1, §7.
[2] P. Addison, M. H. Nguyen, T. Medan, J. Shah, M. T. Manzari, B. McElrone, L. Lalwani, A. More, S. Sharma, H. R. Roth, I. Yang, C. Chen, D. Xu, Y. Cheng, A. Feng, and Z. Xu (2024) C-FedRAG: A confidential federated retrieval-augmented generation system. arXiv preprint arXiv:2412.13163. Cited by: §B.2, §4, Table 1. [2] P. Addison, M. H. Nguyen, T. Medan, J. Shah, M. T. Manzari, B. McElrone, L. Lalwani, A. More, S. Sharma, H. R. Roth, I. Yang, C. Chen, D. Xu, Y. Cheng, A. Feng, and Z. Xu (2024) C-FedRAG: A confidential federated retrieval-augmented generation system. arXiv preprint arXiv:2412.13163. Cited by: §B.2, §4, Table 1.
[3] L. A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M. J. Ryan, M. Jiang, C. Potts, K. Sen, A. G. Dimakis, I. Stoica, D. Klein, M. Zaharia, and O. Khattab (2026) GEPA: Reflective prompt evolution can outperform reinforcement learning. In International Conference on Learning Representations, Cited by: §B.1, §4. [3] L. A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M. J. Ryan, M. Jiang, C. Potts, K. Sen, A. G. Dimakis, I. Stoica, D. Klein, M. Zaharia, and O. Khattab (2026) GEPA: Reflective prompt evolution can outperform reinforcement learning. In International Conference on Learning Representations, Cited by: §B.1, §4.
[4] S. Alam, L. Liu, M. Yan, and M. Zhang (2022) FedRolex: model-heterogeneous federated learning with rolling sub-model extraction. In Advances in Neural Information Processing Systems, Vol. 35. Cited by: §7. [4] S. Alam, L. Liu, M. Yan, and M. Zhang (2022) FedRolex: model-heterogeneous federated learning with rolling sub-model extraction. In Advances in Neural Information Processing Systems, Vol. 35. Cited by: §7.
[5] J. Allen, G. Ferguson, and A. Stent (2001) An architecture for more realistic conversational systems. In Proceedings of the 6th international conference on Intelligent user interfaces, pp. 1–8. Cited by: §1. [5] J. Allen, G. Ferguson, and A. Stent (2001) An architecture for more realistic conversational systems. In Proceedings of the 6th international conference on Intelligent user interfaces, pp. 1–8. Cited by: §1.
[6] Y. I. Alzoubi and A. Mishra (2025) Differential privacy and artificial intelligence: potentials, challenges, and future avenues. EURASIP Journal on Information Security 2025 (1), pp. 18. Cited by: §2. [6] Y. I. Alzoubi and A. Mishra (2025) Differential privacy and artificial intelligence: potentials, challenges, and future avenues. EURASIP Journal on Information Security 2025 (1), pp. 18. Cited by: §2.
[7] M. Andriushchenko, A. Souly, M. Dziemian, D. Duenas, M. Lin, J. Wang, D. Hendrycks, A. Zou, Z. Kolter, M. Fredrikson, et al. (2025) Agentharm: a benchmark for measuring harmfulness of llm agents. In International Conference on Learning Representations, Vol. 2025, pp. 79185–79220. Cited by: §1. [7] M. Andriushchenko, A. Souly, M. Dziemian, D. Duenas, M. Lin, J. Wang, D. Hendrycks, A. Zou, Z. Kolter, M. Fredrikson, et al. (2025) Agentharm: a benchmark for measuring harmfulness of llm agents. In International Conference on Learning Representations, Vol. 2025, pp. 79185–79220. Cited by: §1.
[8] Anthropic (2024) Model context protocol (MCP). Note: https://modelcontextprotocol.ioOpen protocol; announced 25 November 2024 at https://www.anthropic.com/news/model-context-protocol. Accessed 29 September 2026 Cited by: §B.3, §4. [8] Anthropic (2024) Model context protocol (MCP). Note: https://modelcontextprotocol.ioOpen protocol; announced 25 November 2024 at https://www.anthropic.com/news/model-context-protocol. Accessed 29 September 2026 Cited by: §B.3, §4.
[9] J. A. Arjona-Medina, M. Gillhofer, M. Widrich, T. Unterthiner, J. Brandstetter, and S. Hochreiter (2019) Rudder: return decomposition for delayed rewards. Advances in Neural Information Processing Systems 32. Cited by: §6.2. [9] J. A. Arjona-Medina, M. Gillhofer, M. Widrich, T. Unterthiner, J. Brandstetter, and S. Hochreiter (2019) Rudder: return decomposition for delayed rewards. Advances in Neural Information Processing Systems 32. Cited by: §6.2.
[10] H. Babaei Giglou, J. D’Souza, F. Engel, and S. Auer (2025) LLMs4OM: Matching ontologies with large language models. In The Semantic Web: ESWC 2024 Satellite Events, Lecture Notes in Computer Science, Vol. 15344, Cham, pp. 25–35. Cited by: §B.5, §4. [10] H. Babaei Giglou, J. D’Souza, F. Engel, and S. Auer (2025) LLMs4OM: Matching ontologies with large language models. In The Semantic Web: ESWC 2024 Satellite Events, Lecture Notes in Computer Science, Vol. 15344, Cham, pp. 25–35. Cited by: §B.5, §4.
[11] J. Bai, D. Chen, B. Qian, L. Yao, and Y. Li (2024) Federated fine-tuning of large language models under heterogeneous tasks and client resources. In Advances in Neural Information Processing Systems, Vol. 37, pp. 14457–14483. Cited by: §B.1, §4. [11] J. Bai, D. Chen, B. Qian, L. Yao, and Y. Li (2024) Federated fine-tuning of large language models under heterogeneous tasks and client resources. In Advances in Neural Information Processing Systems, Vol. 37, pp. 14457–14483. Cited by: §B.1, §4.
[12] S. Barke, A. Goyal, A. Khare, A. Singh, S. Nath, and C. Bansal (2026) Agentrx: diagnosing ai agent failures from execution trajectories. arXiv preprint arXiv:2602.02475. Cited by: §1. [12] S. Barke, A. Goyal, A. Khare, A. Singh, S. Nath, and C. Bansal (2026) Agentrx: diagnosing ai agent failures from execution trajectories. arXiv preprint arXiv:2602.02475. Cited by: §1.
[13] R. D. Beer (1995) A dynamical systems perspective on agent-environment interaction. Artificial intelligence 72 (1-2), pp. 173–215. Cited by: §1. [13] R. D. Beer (1995) A dynamical systems perspective on agent-environment interaction. Artificial intelligence 72 (1-2), pp. 173–215. Cited by: §1.
[14] J. Bian, L. Wang, L. Zhang, and J. Xu (2025) Lora-fair: federated lora fine-tuning with aggregation and initialization refinement. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3737–3746. Cited by: §2. [14] J. Bian, L. Wang, L. Zhang, and J. Xu (2025) Lora-fair: federated lora fine-tuning with aggregation and initialization refinement. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3737–3746. Cited by: §2.
[15] P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer (2017) Machine learning with adversaries: byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §7. [15] P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer (2017) Machine learning with adversaries: byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §7.
[16] A. Blum, N. Haghtalab, R. L. Phillips, and H. Shao (2021) One for one, or all for all: Equilibria and optimality of collaboration in federated learning. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, pp. 1005–1014. Cited by: §5.3. [16] A. Blum, N. Haghtalab, R. L. Phillips, and H. Shao (2021) One for one, or all for all: Equilibria and optimality of collaboration in federated learning. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, pp. 1005–1014. Cited by: §5.3.
[17] X. Bo, Z. Zhang, Q. Dai, X. Feng, L. Wang, R. Li, X. Chen, and J. Wen (2024) Reflective multi-agent collaboration based on large language models. Advances in Neural Information Processing Systems 37, pp. 138595–138631. Cited by: §2. [17] X. Bo, Z. Zhang, Q. Dai, X. Feng, L. Wang, R. Li, X. Chen, and J. Wen (2024) Reflective multi-agent collaboration based on large language models. Advances in Neural Information Processing Systems 37, pp. 138595–138631. Cited by: §2.
[18] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth (2017) Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1175–1191. Cited by: §B.1, §4, §4, §6.1, §7. [18] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth (2017) Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 1175–1191. Cited by: §B.1, §4, §4, §6.1, §7.
[19] P. Bresciani, A. Perini, P. Giorgini, F. Giunchiglia, and J. Mylopoulos (2004) Tropos: an agent-oriented software development methodology. Autonomous Agents and Multi-Agent Systems 8 (3), pp. 203–236. Cited by: §1. [19] P. Bresciani, A. Perini, P. Giorgini, F. Giunchiglia, and J. Mylopoulos (2004) Tropos: an agent-oriented software development methodology. Autonomous Agents and Multi-Agent Systems 8 (3), pp. 203–236. Cited by: §1.
[20] T. Cai, X. Wang, T. Ma, X. Chen, and D. Zhou (2024) Large language models as tool makers. In International Conference on Learning Representations, Cited by: §B.3. [20] T. Cai, X. Wang, T. Ma, X. Chen, and D. Zhou (2024) Large language models as tool makers. In International Conference on Learning Representations, Cited by: §B.3.
[21] X. Cao, M. Fang, J. Liu, and N. Z. Gong (2021) FLTrust: byzantine-robust federated learning via trust bootstrapping. In Network and Distributed System Security Symposium, External Links: Document Cited by: §7. [21] X. Cao, M. Fang, J. Liu, and N. Z. Gong (2021) FLTrust: byzantine-robust federated learning via trust bootstrapping. In Network and Distributed System Security Symposium, External Links: Document Cited by: §7.
[22] X. Cao, T. Başar, S. Diggavi, Y. C. Eldar, K. B. Letaief, H. V. Poor, and J. Zhang (2023) Communication-efficient distributed learning: an overview. IEEE journal on selected areas in communications 41 (4), pp. 851–873. Cited by: §2. [22] X. Cao, T. Başar, S. Diggavi, Y. C. Eldar, K. B. Letaief, H. V. Poor, and J. Zhang (2023) Communication-efficient distributed learning: an overview. IEEE journal on selected areas in communications 41 (4), pp. 851–873. Cited by: §2.
[23] H. Chae, S. Kim, J. Cho, S. Kim, S. Moon, G. Hwangbo, D. Lim, M. Kim, Y. Hwang, M. Gwak, D. Choi, M. Kang, G. Im, B. Cho, H. Kim, J. H. Han, T. Kwon, M. Kim, B. Kwak, D. Kang, and J. Yeo (2025) Web-Shepherd: Advancing PRMs for reinforcing web agents. In Advances in Neural Information Processing Systems, Vol. 38, pp. 63314–63356. Cited by: §B.4, §4. [23] H. Chae, S. Kim, J. Cho, S. Kim, S. Moon, G. Hwangbo, D. Lim, M. Kim, Y. Hwang, M. Gwak, D. Choi, M. Kang, G. Im, B. Cho, H. Kim, J. H. Han, T. Kwon, M. Kim, B. Kwak, D. Kang, and J. Yeo (2025) Web-Shepherd: Advancing PRMs for reinforcing web agents. In Advances in Neural Information Processing Systems, Vol. 38, pp. 63314–63356. Cited by: §B.4, §4.
[24] A. Chakraborty, C. Dahal, and V. Gupta (2025) Federated retrieval-augmented generation: A systematic mapping study. In Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, pp. 7362–7374. Cited by: §B.2, §4. [24] A. Chakraborty, C. Dahal, and V. Gupta (2025) Federated retrieval-augmented generation: A systematic mapping study. In Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, pp. 7362–7374. Cited by: §B.2, §4.
[25] A. Chakraborty, N. Trieu, and V. Gupta (2026) Typed federated artifacts for the agentic web: Sharing tool-routing knowledge across frozen, heterogeneous LLM agents. arXiv preprint arXiv:2609.06815. Cited by: §B.3, §B.3, Table 4, §3.2, §4, §4, Table 1. [25] A. Chakraborty, N. Trieu, and V. Gupta (2026) Typed federated artifacts for the agentic web: Sharing tool-routing knowledge across frozen, heterogeneous LLM agents. arXiv preprint arXiv:2609.06815. Cited by: §B.3, §B.3, Table 4, §3.2, §4, §4, Table 1.
[26] F. Chang, G. Zhou, C. Zhang, K. Ding, W. Cheng, and F. Chang (2021) A maintenance decision-making oriented collaborative cross-organization knowledge sharing blockchain network for complex multi-component systems. Journal of cleaner production 282, pp. 124541. Cited by: §2. [26] F. Chang, G. Zhou, C. Zhang, K. Ding, W. Cheng, and F. Chang (2021) A maintenance decision-making oriented collaborative cross-organization knowledge sharing blockchain network for complex multi-component systems. Journal of cleaner production 282, pp. 124541. Cited by: §2.
[27] T. Che, J. Liu, Y. Zhou, J. Ren, J. Zhou, V. Sheng, H. Dai, and D. Dou (2023) Federated learning of large language models with parameter-efficient prompt tuning and adaptive optimization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7871–7888. Cited by: §2. [27] T. Che, J. Liu, Y. Zhou, J. Ren, J. Zhou, V. Sheng, H. Dai, and D. Dou (2023) Federated learning of large language models with parameter-efficient prompt tuning and adaptive optimization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7871–7888. Cited by: §2.
[28] A. Chen, Z. Liu, J. Zhang, A. Prabhakar, Z. Liu, S. Heinecke, S. Savarese, V. Zhong, and C. Xiong (2026) Test-time adaptation for llm agents via environment interaction. In International Conference on Learning Representations, Vol. 2026, pp. 39256–39288. Cited by: §1. [28] A. Chen, Z. Liu, J. Zhang, A. Prabhakar, Z. Liu, S. Heinecke, S. Savarese, V. Zhong, and C. Xiong (2026) Test-time adaptation for llm agents via environment interaction. In International Conference on Learning Representations, Vol. 2026, pp. 39256–39288. Cited by: §1.
[29] C. Chen, K. Zhu, Z. Chen, Z. Zhou, S. Diao, Y. Lu, T. Li, M. Li, and D. Song (2026) Is decentralized llm agent rl robust to heterogeneity? an asymmetric tale. arXiv preprint. External Links: Link Cited by: §B.1, Table 4, §4, Table 1, §5.3. [29] C. Chen, K. Zhu, Z. Chen, Z. Zhou, S. Diao, Y. Lu, T. Li, M. Li, and D. Song (2026) Is decentralized llm agent rl robust to heterogeneity? an asymmetric tale. arXiv preprint. External Links: Link Cited by: §B.1, Table 4, §4, Table 1, §5.3.
[30] M. Chen, R. Jin, W. Deng, Y. Chen, Z. Huang, H. Yu, and X. Li (2025) Can textual gradient work in federated learning?. In International Conference on Learning Representations, Cited by: §B.1, Table 4, §4, Table 1. [30] M. Chen, R. Jin, W. Deng, Y. Chen, Z. Huang, H. Yu, and X. Li (2025) Can textual gradient work in federated learning?. In International Conference on Learning Representations, Cited by: §B.1, Table 4, §4, Table 1.
[31] M. Chen, W. Zhang, Z. Yuan, Y. Jia, and H. Chen (2021) FedE: Embedding knowledge graphs in federated setting. In Proceedings of the 10th International Joint Conference on Knowledge Graphs, pp. 80–88. Cited by: §B.5, §C.1, §4, Table 1, Table 2. [31] M. Chen, W. Zhang, Z. Yuan, Y. Jia, and H. Chen (2021) FedE: Embedding knowledge graphs in federated setting. In Proceedings of the 10th International Joint Conference on Knowledge Graphs, pp. 80–88. Cited by: §B.5, §C.1, §4, Table 1, Table 2.
[32] M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor (2021) Distributed learning in wireless networks: recent progress and future challenges. IEEE journal on selected areas in communications 39 (12), pp. 3579–3605. Cited by: §2. [32] M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor (2021) Distributed learning in wireless networks: recent progress and future challenges. IEEE journal on selected areas in communications 39 (12), pp. 3579–3605. Cited by: §2.
[33] M. Chen, N. Shlezinger, H. V. Poor, Y. C. Eldar, and S. Cui (2021) Communication-efficient federated learning. Proceedings of the National Academy of Sciences 118 (17), pp. e2024789118. Cited by: §2. [33] M. Chen, N. Shlezinger, H. V. Poor, Y. C. Eldar, and S. Cui (2021) Communication-efficient federated learning. Proceedings of the National Academy of Sciences 118 (17), pp. e2024789118. Cited by: §2.
[34] N. Chen, Y. Tong, Y. Yang, Y. He, X. Zhang, Z. Qingyun, Q. Wang, and B. He (2026) Diversity collapse in multi-agent llm systems: structural coupling and collective failure in open-ended idea generation. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 251–306. Cited by: §2. [34] N. Chen, Y. Tong, Y. Yang, Y. He, X. Zhang, Z. Qingyun, Q. Wang, and B. He (2026) Diversity collapse in multi-agent llm systems: structural coupling and collective failure in open-ended idea generation. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 251–306. Cited by: §2.
[35] X. Chen, Y. Shi, Q. Lan, Y. Qiu, M. Wang, X. Gu, and Y. Yan (2025) Fed-SE: Federated self-evolution for privacy-constrained multi-environment LLM agents. arXiv preprint arXiv:2512.08870. Cited by: §B.1, §B.1, Table 4, §3.4, §4, §4, Table 1. [35] X. Chen, Y. Shi, Q. Lan, Y. Qiu, M. Wang, X. Gu, and Y. Yan (2025) Fed-SE: Federated self-evolution for privacy-constrained multi-environment LLM agents. arXiv preprint arXiv:2512.08870. Cited by: §B.1, §B.1, Table 4, §3.4, §4, §4, Table 1.
[36] Y. Chen, J. Yoon, D. S. Sachan, Q. Wang, V. Cohen-Addad, M. Bateni, C. Lee, and T. Pfister (2024) Re-invoke: tool invocation rewriting for zero-shot tool retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 4705–4726. Cited by: §1. [36] Y. Chen, J. Yoon, D. S. Sachan, Q. Wang, V. Cohen-Addad, M. Bateni, C. Lee, and T. Pfister (2024) Re-invoke: tool invocation rewriting for zero-shot tool retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 4705–4726. Cited by: §1.
[37] Y. Cheng, H. Ye, H. H. Li, J. Sun, and Y. Chen (2026) Privact: internalizing contextual privacy preservation via multi-agent preference training. arXiv preprint arXiv:2602.13840. Cited by: §1. [37] Y. Cheng, H. Ye, H. H. Li, J. Sun, and Y. Chen (2026) Privact: internalizing contextual privacy preservation via multi-agent preference training. arXiv preprint arXiv:2602.13840. Cited by: §1.
[38] Y. J. Cho, L. Liu, Z. Xu, A. Fahrezi, and G. Joshi (2024) Heterogeneous LoRA for federated fine-tuning of on-device foundation models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA, pp. 12903–12913. Cited by: §B.1, §4. [38] Y. J. Cho, L. Liu, Z. Xu, A. Fahrezi, and G. Joshi (2024) Heterogeneous LoRA for federated fine-tuning of on-device foundation models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA, pp. 12903–12913. Cited by: §B.1, §4.
[39] P. F. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei (2017) Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, Vol. 30, pp. 4299–4307. Cited by: §B.4, §7. [39] P. F. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei (2017) Deep reinforcement learning from human preferences. In Advances in Neural Information Processing Systems, Vol. 30, pp. 4299–4307. Cited by: §B.4, §7.
[40] J. A. G. Coria, J. A. Castellanos-Garzón, and J. M. Corchado (2014) Intelligent business processes composition based on multi-agent systems. Expert Systems with Applications 41 (4), pp. 1189–1205. Cited by: §1. [40] J. A. G. Coria, J. A. Castellanos-Garzón, and J. M. Corchado (2014) Intelligent business processes composition based on multi-agent systems. Expert Systems with Applications 41 (4), pp. 1189–1205. Cited by: §1.
[41] Y. Deng, F. Lyu, J. Ren, Y. Chen, P. Yang, Y. Zhou, and Y. Zhang (2021) Fair: quality-aware federated learning with precise user incentive and model aggregation. In IEEE INFOCOM 2021-IEEE Conference on Computer Communications, pp. 1–10. Cited by: §1. [41] Y. Deng, F. Lyu, J. Ren, Y. Chen, P. Yang, Y. Zhou, and Y. Zhang (2021) Fair: quality-aware federated learning with precise user incentive and model aggregation. In IEEE INFOCOM 2021-IEEE Conference on Computer Communications, pp. 1–10. Cited by: §1.
[42] Z. Deng, W. Ma, Q. Han, W. Zhou, X. Zhu, S. Wen, and Y. Xiang (2025) Exploring deepseek: a survey on advances, applications, challenges and future directions. IEEE/CAA Journal of Automatica Sinica 12 (5), pp. 872–893. Cited by: §1. [42] Z. Deng, W. Ma, Q. Han, W. Zhou, X. Zhu, S. Wen, and Y. Xiang (2025) Exploring deepseek: a survey on advances, applications, challenges and future directions. IEEE/CAA Journal of Automatica Sinica 12 (5), pp. 872–893. Cited by: §1.
[43] E. Diao, J. Ding, and V. Tarokh (2021) HeteroFL: computation and communication efficient federated learning for heterogeneous clients. In International Conference on Learning Representations, Cited by: §7. [43] E. Diao, J. Ding, and V. Tarokh (2021) HeteroFL: computation and communication efficient federated learning for heterogeneous clients. In International Conference on Learning Representations, Cited by: §7.
[44] K. Donahue and J. Kleinberg (2021) Model-sharing games: Analyzing federated learning under voluntary participation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, pp. 5303–5311. Cited by: §5.3. [44] K. Donahue and J. Kleinberg (2021) Model-sharing games: Analyzing federated learning under voluntary participation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, pp. 5303–5311. Cited by: §5.3.
[45] C. Dong, Y. Li, H. Gong, M. Chen, J. Li, Y. Shen, and M. Yang (2022) A survey of natural language generation. ACM computing surveys 55 (8), pp. 1–38. Cited by: §1. [45] C. Dong, Y. Li, H. Gong, M. Chen, J. Li, Y. Shen, and M. Yang (2022) A survey of natural language generation. ACM computing surveys 55 (8), pp. 1–38. Cited by: §1.
[46] A. Doshi, Y. Hong, C. Xu, E. Kang, A. Kapravelos, and C. Kästner (2026) Towards verifiably safe tool use for llm agents. In Proceedings of the IEEE/ACM 48th International Conference on Software Engineering, pp. 201–205. Cited by: §1. [46] A. Doshi, Y. Hong, C. Xu, E. Kang, A. Kapravelos, and C. Kästner (2026) Towards verifiably safe tool use for llm agents. In Proceedings of the IEEE/ACM 48th International Conference on Software Engineering, pp. 201–205. Cited by: §1.
[47] C. Dwork and A. Roth (2014) The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science 9 (3–4), pp. 211–487. Cited by: §3.3, §3.3. [47] C. Dwork and A. Roth (2014) The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science 9 (3–4), pp. 211–487. Cited by: §3.3, §3.3.
[48] J. Euzenat and P. Shvaiko (2013) Ontology matching. 2 edition, Springer, Berlin, Heidelberg. External Links: ISBN 978-3-642-38720-3 Cited by: §B.5, §4. [48] J. Euzenat and P. Shvaiko (2013) Ontology matching. 2 edition, Springer, Berlin, Heidelberg. External Links: ISBN 978-3-642-38720-3 Cited by: §B.5, §4.
[49] A. Fallah, A. Mokhtari, and A. Ozdaglar (2020) Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. Advances in neural information processing systems 33, pp. 3557–3568. Cited by: §7. [49] A. Fallah, A. Mokhtari, and A. Ozdaglar (2020) Personalized federated learning with theoretical guarantees: a model-agnostic meta-learning approach. Advances in neural information processing systems 33, pp. 3557–3568. Cited by: §7.
[50] F. X. Fan, Y. Ma, Z. Dai, W. Jing, C. Tan, and B. K. H. Low (2021) Fault-tolerant federated reinforcement learning with theoretical guarantee. In Advances in Neural Information Processing Systems, Vol. 34, pp. 1007–1021. Cited by: §B.1. [50] F. X. Fan, Y. Ma, Z. Dai, W. Jing, C. Tan, and B. K. H. Low (2021) Fault-tolerant federated reinforcement learning with theoretical guarantee. In Advances in Neural Information Processing Systems, Vol. 34, pp. 1007–1021. Cited by: §B.1.
[51] F. X. Fan, C. Tan, Y. Ong, R. Wattenhofer, and W. Ooi (2025) FedRLHF: A convergence-guaranteed federated framework for privacy-preserving and personalized RLHF. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, Detroit, Michigan, USA, pp. 713–721. Cited by: §B.4, Table 4, §4, Table 1. [51] F. X. Fan, C. Tan, Y. Ong, R. Wattenhofer, and W. Ooi (2025) FedRLHF: A convergence-guaranteed federated framework for privacy-preserving and personalized RLHF. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, Detroit, Michigan, USA, pp. 713–721. Cited by: §B.4, Table 4, §4, Table 1.
[52] T. Fan, G. Ma, Y. Kang, H. Gu, Y. Song, L. Fan, K. Chen, and Q. Yang (2025) FedMKT: Federated mutual knowledge transfer for large and small language models. In Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, UAE, pp. 243–255. Cited by: §5.1. [52] T. Fan, G. Ma, Y. Kang, H. Gu, Y. Song, L. Fan, K. Chen, and Q. Yang (2025) FedMKT: Federated mutual knowledge transfer for large and small language models. In Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, UAE, pp. 243–255. Cited by: §5.1.
[53] H. Gao and Y. Zhang (2024) INMS: Memory sharing for large language model based agents. arXiv preprint arXiv:2404.09982. Cited by: §B.2, §4. [53] H. Gao and Y. Zhang (2024) INMS: Memory sharing for large language model based agents. arXiv preprint arXiv:2404.09982. Cited by: §B.2, §4.
[54] J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller (2020) Inverting gradients-how easy is it to break privacy in federated learning?. Advances in neural information processing systems 33, pp. 16937–16947. Cited by: §6.1. [54] J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller (2020) Inverting gradients-how easy is it to break privacy in federated learning?. Advances in neural information processing systems 33, pp. 16937–16947. Cited by: §6.1.
[55] R. C. Geyer, T. Klein, and M. Nabi (2017) Differentially private federated learning: a client level perspective. arXiv preprint arXiv:1712.07557. Cited by: §1, §2. [55] R. C. Geyer, T. Klein, and M. Nabi (2017) Differentially private federated learning: a client level perspective. arXiv preprint arXiv:1712.07557. Cited by: §1, §2.
[56] A. Ghorbani and J. Zou (2019) Data Shapley: Equitable valuation of data for machine learning. In Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 97, pp. 2242–2251. Cited by: §5.3. [56] A. Ghorbani and J. Zou (2019) Data Shapley: Equitable valuation of data for machine learning. In Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 97, pp. 2242–2251. Cited by: §5.3.
[57] A. Ghosh, J. Chung, D. Yin, and K. Ramchandran (2020) An efficient framework for clustered federated learning. In Advances in Neural Information Processing Systems, Vol. 33, pp. 19586–19597. Cited by: §5.3. [57] A. Ghosh, J. Chung, D. Yin, and K. Ramchandran (2020) An efficient framework for clustered federated learning. In Advances in Neural Information Processing Systems, Vol. 33, pp. 19586–19597. Cited by: §5.3.
[58] U. Gnewuch, S. Morana, and A. Mädche (2017) Towards designing cooperative and social conversational agents for customer service. In Proceedings of the 38th International Conference on Information Systems (ICIS), Cited by: §1. [58] U. Gnewuch, S. Morana, and A. Mädche (2017) Towards designing cooperative and social conversational agents for customer service. In Proceedings of the 38th International Conference on Information Systems (ICIS), Cited by: §1.
[59] P. Guo, S. Zeng, Y. Wang, H. Fan, F. Wang, and L. Qu (2025) Selective aggregation for low-rank adaptation in federated learning. In International Conference on Learning Representations, Cited by: §B.1, §4. [59] P. Guo, S. Zeng, Y. Wang, H. Fan, F. Wang, and L. Qu (2025) Selective aggregation for low-rank adaptation in federated learning. In International Conference on Learning Representations, Cited by: §B.1, §4.
[60] T. Guo, X. Chen, Y. Wang, R. Chang, S. Pei, N. V. Chawla, O. Wiest, and X. Zhang (2024) Large language model based multi-agents: a survey of progress and challenges. arXiv preprint arXiv:2402.01680. Cited by: §1, §2. [60] T. Guo, X. Chen, Y. Wang, R. Chang, S. Pei, N. V. Chawla, O. Wiest, and X. Zhang (2024) Large language model based multi-agents: a survey of progress and challenges. arXiv preprint arXiv:2402.01680. Cited by: §1, §2.
[61] T. Guo, S. Guo, J. Wang, X. Tang, and W. Xu (2024) PromptFL: Let federated participants cooperatively learn prompts instead of models – federated learning in age of foundation model. IEEE Transactions on Mobile Computing 23 (5), pp. 5179–5194. Cited by: §B.1. [61] T. Guo, S. Guo, J. Wang, X. Tang, and W. Xu (2024) PromptFL: Let federated participants cooperatively learn prompts instead of models – federated learning in age of foundation model. IEEE Transactions on Mobile Computing 23 (5), pp. 5179–5194. Cited by: §B.1.
[62] T. Guo (2018) Cloud-based or on-device: an empirical study of mobile deep inference. In 2018 IEEE International Conference on Cloud Engineering (IC2E), pp. 184–190. Cited by: §2. [62] T. Guo (2018) Cloud-based or on-device: an empirical study of mobile deep inference. In 2018 IEEE International Conference on Cloud Engineering (IC2E), pp. 184–190. Cited by: §2.
[63] Y. Guo, J. Lin, H. Wang, Y. Han, S. Hu, Z. Ni, L. Wang, and M. Chen (2026) Se-agent: self-evolution trajectory optimization in multi-step reasoning with llm-based agents. Advances in Neural Information Processing Systems 38, pp. 116314–116341. Cited by: §1. [63] Y. Guo, J. Lin, H. Wang, Y. Han, S. Hu, Z. Ni, L. Wang, and M. Chen (2026) Se-agent: self-evolution trajectory optimization in multi-step reasoning with llm-based agents. Advances in Neural Information Processing Systems 38, pp. 116314–116341. Cited by: §1.
[64] S. Gupta, Y. Huang, Z. Zhong, T. Gao, K. Li, and D. Chen (2022) Recovering private text in federated learning of language models. In Advances in Neural Information Processing Systems, Vol. 35, pp. 8130–8143. Cited by: §B.1. [64] S. Gupta, Y. Huang, Z. Zhong, T. Gao, K. Li, and D. Chen (2022) Recovering private text in federated learning of language models. In Advances in Neural Information Processing Systems, Vol. 35, pp. 8130–8143. Cited by: §B.1.
[65] B. Ha, J. Bae, Y. T. Park, and S. Kang (2006) Development of process execution rules for workload balancing on agents. Data & Knowledge Engineering 56 (1), pp. 64–84. Cited by: §1. [65] B. Ha, J. Bae, Y. T. Park, and S. Kang (2006) Development of process execution rules for workload balancing on agents. Data & Knowledge Engineering 56 (1), pp. 64–84. Cited by: §1.
[66] E. Han, D. Yin, and H. Zhang (2023) Bots with feelings: should ai agents express positive emotion in customer service?. Information Systems Research 34 (3), pp. 1296–1311. Cited by: §1. [66] E. Han, D. Yin, and H. Zhang (2023) Bots with feelings: should ai agents express positive emotion in customer service?. Information Systems Research 34 (3), pp. 1296–1311. Cited by: §1.
[67] S. Hao, T. Liu, Z. Wang, and Z. Hu (2023) Toolkengpt: augmenting frozen language models with massive tools via tool embeddings. Advances in neural information processing systems 36, pp. 45870–45894. Cited by: §7. [67] S. Hao, T. Liu, Z. Wang, and Z. Hu (2023) Toolkengpt: augmenting frozen language models with massive tools via tool embeddings. Advances in neural information processing systems 36, pp. 45870–45894. Cited by: §7.
[68] A. Harutyunyan, W. Dabney, T. Mesnard, M. Gheshlaghi Azar, B. Piot, N. Heess, H. P. van Hasselt, G. Wayne, S. Singh, D. Precup, et al. (2019) Hindsight credit assignment. Advances in neural information processing systems 32. Cited by: §6.2. [68] A. Harutyunyan, W. Dabney, T. Mesnard, M. Gheshlaghi Azar, B. Piot, N. Heess, H. P. van Hasselt, G. Wayne, S. Singh, D. Precup, et al. (2019) Hindsight credit assignment. Advances in neural information processing systems 32. Cited by: §6.2.
[69] J. He, C. Treude, and D. Lo (2025) Llm-based multi-agent systems for software engineering: literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology 34 (5), pp. 1–30. Cited by: §2. [69] J. He, C. Treude, and D. Lo (2025) Llm-based multi-agent systems for software engineering: literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology 34 (5), pp. 1–30. Cited by: §2.
[70] Y. He, Y. Kang, L. Fan, and Q. Yang (2024) FedEval-LLM: Federated evaluation of large language models on downstream tasks with collective wisdom. arXiv preprint arXiv:2404.12273. Cited by: §B.4, §4. [70] Y. He, Y. Kang, L. Fan, and Q. Yang (2024) FedEval-LLM: Federated evaluation of large language models on downstream tasks with collective wisdom. arXiv preprint arXiv:2404.12273. Cited by: §B.4, §4.
[71] S. Hertling and H. Paulheim (2023) OLaLa: Ontology matching with large language models. In Proceedings of the 12th Knowledge Capture Conference 2023, pp. 131–139. Cited by: §B.5. [71] S. Hertling and H. Paulheim (2023) OLaLa: Ontology matching with large language models. In Proceedings of the 12th Knowledge Capture Conference 2023, pp. 131–139. Cited by: §B.5.
[72] S. Hong, Y. Lin, B. Liu, B. Liu, B. Wu, C. Zhang, D. Li, J. Chen, J. Zhang, J. Wang, et al. (2025) Data interpreter: an llm agent for data science. In Findings of the association for computational linguistics: ACL 2025, pp. 19796–19821. Cited by: §1. [72] S. Hong, Y. Lin, B. Liu, B. Liu, B. Wu, C. Zhang, D. Li, J. Chen, J. Zhang, J. Wang, et al. (2025) Data interpreter: an llm agent for data science. In Findings of the association for computational linguistics: ACL 2025, pp. 19796–19821. Cited by: §1.
[73] S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, C. Ran, L. Xiao, C. Wu, and J. Schmidhuber (2024) MetaGPT: meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations, Cited by: §7. [73] S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, C. Ran, L. Xiao, C. Wu, and J. Schmidhuber (2024) MetaGPT: meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations, Cited by: §7.
[74] X. Hou, Y. Zhao, S. Wang, and H. Wang (2026) Model context protocol (MCP): Landscape, security threats, and future research directions. ACM Transactions on Software Engineering and Methodology 35 (10), pp. 302:1–302:37. Cited by: §B.3, §4. [74] X. Hou, Y. Zhao, S. Wang, and H. Wang (2026) Model context protocol (MCP): Landscape, security threats, and future research directions. ACM Transactions on Software Engineering and Methodology 35 (10), pp. 302:1–302:37. Cited by: §B.3, §4.
[75] Y. Hou (2026) FedWorld: Scope-aware federation of agent world models. arXiv preprint arXiv:2608.01561. Cited by: §B.5, §B.6, Table 4, §C.1, §3.2, §4, §4, Table 1, §5.3, Table 2. [75] Y. Hou (2026) FedWorld: Scope-aware federation of agent world models. arXiv preprint arXiv:2608.01561. Cited by: §B.5, §B.6, Table 4, §C.1, §3.2, §4, §4, Table 1, §5.3, Table 2.
[76] W. Huang, M. Ye, Z. Shi, G. Wan, H. Li, B. Du, and Q. Yang (2024) Federated learning for generalization, robustness, fairness: a survey and benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (12), pp. 9387–9406. Cited by: §2. [76] W. Huang, M. Ye, Z. Shi, G. Wan, H. Li, B. Du, and Q. Yang (2024) Federated learning for generalization, robustness, fairness: a survey and benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence 46 (12), pp. 9387–9406. Cited by: §2.
[77] W. Huang, P. Abbeel, D. Pathak, and I. Mordatch (2022) Language models as zero-shot planners: extracting actionable knowledge for embodied agents. In International conference on machine learning, pp. 9118–9147. Cited by: §7. [77] W. Huang, P. Abbeel, D. Pathak, and I. Mordatch (2022) Language models as zero-shot planners: extracting actionable knowledge for embodied agents. In International conference on machine learning, pp. 9118–9147. Cited by: §7.
[78] Z. Huang, X. Yuan, Y. Ju, J. Zhao, and K. Liu (2025) Reinforced internal-external knowledge synergistic reasoning for efficient adaptive search agent. arXiv preprint arXiv:2505.07596. Cited by: §1. [78] Z. Huang, X. Yuan, Y. Ju, J. Zhao, and K. Liu (2025) Reinforced internal-external knowledge synergistic reasoning for efficient adaptive search agent. arXiv preprint arXiv:2505.07596. Cited by: §1.
[79] D. Isern, D. Sánchez, and A. Moreno (2010) Agents applied in health care: a review. International journal of medical informatics 79 (3), pp. 145–166. Cited by: §1. [79] D. Isern, D. Sánchez, and A. Moreno (2010) Agents applied in health care: a review. International journal of medical informatics 79 (3), pp. 145–166. Cited by: §1.
[80] B. Jayaraman, L. Wang, D. Evans, and Q. Gu (2018) Distributed learning without distress: privacy-preserving empirical risk minimization. Advances in neural information processing systems 31. Cited by: §2. [80] B. Jayaraman, L. Wang, D. Evans, and Q. Gu (2018) Distributed learning without distress: privacy-preserving empirical risk minimization. Advances in neural information processing systems 31. Cited by: §2.
[81] N. R. Jennings, T. J. Norman, P. Faratin, P. O’Brien, and B. Odgers (2000) Autonomous agents for business process management. Applied Artificial Intelligence 14 (2), pp. 145–189. Cited by: §1. [81] N. R. Jennings, T. J. Norman, P. Faratin, P. O’Brien, and B. Odgers (2000) Autonomous agents for business process management. Applied Artificial Intelligence 14 (2), pp. 145–189. Cited by: §1.
[82] W. Jeong and S. J. Hwang (2022) Factorized-fl: personalized federated learning with parameter factorization & similarity matching. In Advances in Neural Information Processing Systems, Vol. 35. Cited by: §7. [82] W. Jeong and S. J. Hwang (2022) Factorized-fl: personalized federated learning with parameter factorization & similarity matching. In Advances in Neural Information Processing Systems, Vol. 35. Cited by: §7.
[83] Z. Ji, D. Wu, W. Jiang, P. Ma, Z. Li, Y. Gao, S. Wang, and Y. Li (2026) Taming various privilege escalation in llm-based agent systems: a mandatory access control framework. arXiv preprint arXiv:2601.11893. Cited by: §2. [83] Z. Ji, D. Wu, W. Jiang, P. Ma, Z. Li, Y. Gao, S. Wang, and Y. Li (2026) Taming various privilege escalation in llm-based agent systems: a mandatory access control framework. arXiv preprint arXiv:2601.11893. Cited by: §2.
[84] H. Jin, Y. Peng, W. Yang, S. Wang, and Z. Zhang (2022) Federated reinforcement learning with environment heterogeneity. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 151, pp. 18–37. Cited by: §B.1. [84] H. Jin, Y. Peng, W. Yang, S. Wang, and Z. Zhang (2022) Federated reinforcement learning with environment heterogeneity. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 151, pp. 18–37. Cited by: §B.1.
[85] Y. Kang, H. Gu, X. Tang, Y. He, Y. Zhang, J. He, Y. Han, L. Fan, K. Chen, and Q. Yang (2024) Optimizing privacy, utility, and efficiency in a constrained multi-objective federated learning framework. ACM Transactions on Intelligent Systems and Technology 15 (6), pp. 134:1–134:33. Cited by: §3.3. [85] Y. Kang, H. Gu, X. Tang, Y. He, Y. Zhang, J. He, Y. Han, L. Fan, K. Chen, and Q. Yang (2024) Optimizing privacy, utility, and efficiency in a constrained multi-objective federated learning framework. ACM Transactions on Intelligent Systems and Technology 15 (6), pp. 134:1–134:33. Cited by: §3.3.
[86] S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh (2020) SCAFFOLD: Stochastic controlled averaging for federated learning. In Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 119, pp. 5132–5143. Cited by: §6. [86] S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh (2020) SCAFFOLD: Stochastic controlled averaging for federated learning. In Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 119, pp. 5132–5143. Cited by: §6.
[87] M. J. Khan, B. Fang, G. Cimino, S. Cirillo, L. Yang, and D. Zhao (2024) Privacy-preserving artificial intelligence on edge devices: a homomorphic encryption approach. In 2024 IEEE international conference on web services (ICWS), pp. 395–405. Cited by: §2. [87] M. J. Khan, B. Fang, G. Cimino, S. Cirillo, L. Yang, and D. Zhao (2024) Privacy-preserving artificial intelligence on edge devices: a homomorphic encryption approach. In 2024 IEEE international conference on web services (ICWS), pp. 395–405. Cited by: §2.
[88] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts (2024) DSPy: Compiling declarative language model calls into state-of-the-art pipelines. In International Conference on Learning Representations, Cited by: §B.1, §3.4, §4. [88] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, H. Miller, M. Zaharia, and C. Potts (2024) DSPy: Compiling declarative language model calls into state-of-the-art pipelines. In International Conference on Learning Representations, Cited by: §B.1, §3.4, §4.
[89] S. Khodadadian, P. Sharma, G. Joshi, and S. T. Maguluri (2022) Federated reinforcement learning: Linear speedup under Markovian sampling. In Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 162, pp. 10997–11057. Cited by: §B.1. [89] S. Khodadadian, P. Sharma, G. Joshi, and S. T. Maguluri (2022) Federated reinforcement learning: Linear speedup under Markovian sampling. In Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 162, pp. 10997–11057. Cited by: §B.1.
[90] S. Kim, B. Cho, J. Oh, S. Chung, and S. Yun (2026) Rethinking personalized reward modeling for LLMs under preference heterogeneity via group-debiased federated learning. arXiv preprint arXiv:2608.01556. Cited by: §B.4, §B.4, Table 4, §C.1, §4, Table 1, Table 2. [90] S. Kim, B. Cho, J. Oh, S. Chung, and S. Yun (2026) Rethinking personalized reward modeling for LLMs under preference heterogeneity via group-debiased federated learning. arXiv preprint arXiv:2608.01556. Cited by: §B.4, §B.4, Table 4, §C.1, §4, Table 1, Table 2.
[91] N. Kokash, L. Wang, T. H. Gillespie, A. S. Z. Belloum, P. Grosso, S. Quinney, L. Li, and B. de Bono (2026) Ontology- and LLM-based data harmonization for federated learning in healthcare. Frontiers in Digital Health 8, pp. 1756555. Cited by: §B.5. [91] N. Kokash, L. Wang, T. H. Gillespie, A. S. Z. Belloum, P. Grosso, S. Quinney, L. Li, and B. de Bono (2026) Ontology- and LLM-based data harmonization for federated learning in healthcare. Frontiers in Digital Health 8, pp. 1756555. Cited by: §B.5.
[92] M. Kou, X. Xia, Z. Wang, I. Khalil, R. Luo, J. Zhou, and M. Xue (2026) WinFLoRA: Incentivizing client-adaptive aggregation in federated LoRA under privacy heterogeneity. In Proceedings of the ACM Web Conference 2026, pp. 5241–5252. Cited by: §5.3. [92] M. Kou, X. Xia, Z. Wang, I. Khalil, R. Luo, J. Zhou, and M. Xue (2026) WinFLoRA: Incentivizing client-adaptive aggregation in federated LoRA under privacy heterogeneity. In Proceedings of the ACM Web Conference 2026, pp. 5241–5252. Cited by: §5.3.
[93] W. Kuang, B. Qian, Z. Li, D. Chen, D. Gao, X. Pan, Y. Xie, Y. Li, B. Ding, and J. Zhou (2024) Federatedscope-llm: a comprehensive package for fine-tuning large language models in federated learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5260–5271. Cited by: §2. [93] W. Kuang, B. Qian, Z. Li, D. Chen, D. Gao, X. Pan, Y. Xie, Y. Li, B. Ding, and J. Zhou (2024) Federatedscope-llm: a comprehensive package for fine-tuning large language models in federated learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5260–5271. Cited by: §2.
[94] M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh (2023) Reward design with language models. In International Conference on Learning Representations, Cited by: §B.4, §4. [94] M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh (2023) Reward design with language models. In International Conference on Learning Representations, Cited by: §B.4, §4.
[95] F. Lai, Y. Dai, S. Singapuram, J. Liu, X. Zhu, H. Madhyastha, and M. Chowdhury (2022) Fedscale: benchmarking model and system performance of federated learning at scale. In International conference on machine learning, pp. 11814–11827. Cited by: §6.3, §7. [95] F. Lai, Y. Dai, S. Singapuram, J. Liu, X. Zhu, H. Madhyastha, and M. Chowdhury (2022) Fedscale: benchmarking model and system performance of federated learning at scale. In International conference on machine learning, pp. 11814–11827. Cited by: §6.3, §7.
[96] F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury (2021) Oort: efficient federated learning via guided participant selection. In 15th $\{$ USENIX $\}$ Symposium on Operating Systems Design and Implementation ( $\{$ OSDI $\}$ 21), pp. 19–35. Cited by: §6. [96] F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury (2021) Oort: efficient federated learning via guided participant selection. In 15th {\{USENIX}\} Symposium on Operating Systems Design and Implementation ({\{OSDI}\} 21), pp. 19–35. Cited by: §6.
[97] L. Laranjo, A. G. Dunn, H. L. Tong, A. B. Kocaballi, J. Chen, R. Bashir, D. Surian, B. Gallego, F. Magrabi, A. Y. Lau, et al. (2018) Conversational agents in healthcare: a systematic review. Journal of the american medical informatics association 25 (9), pp. 1248–1258. Cited by: §1. [97] L. Laranjo, A. G. Dunn, H. L. Tong, A. B. Kocaballi, J. Chen, R. Bashir, D. Surian, B. Gallego, F. Magrabi, A. Y. Lau, et al. (2018) Conversational agents in healthcare: a systematic review. Journal of the american medical informatics association 25 (9), pp. 1248–1258. Cited by: §1.
[98] A. Li, Y. Xie, S. Li, F. Tsung, B. Ding, and Y. Li (2025) Agent-oriented planning in multi-agent systems. In International Conference on Learning Representations, Vol. 2025, pp. 19495–19517. Cited by: §2. [98] A. Li, Y. Xie, S. Li, F. Tsung, B. Ding, and Y. Li (2025) Agent-oriented planning in multi-agent systems. In International Conference on Learning Representations, Vol. 2025, pp. 19495–19517. Cited by: §2.
[99] D. Li and J. Wang (2019) FedMD: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581. Cited by: §5.1. [99] D. Li and J. Wang (2019) FedMD: heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581. Cited by: §5.1.
[100] G. Li, H. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem (2023) CAMEL: communicative agents for ”mind” exploration of large language model society. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, pp. 51991–52008. External Links: Document, Link Cited by: §3.4. [100] G. Li, H. Hammoud, H. Itani, D. Khizbullin 和 B. Ghanem (2023) CAMEL：用于大型语言模型社会“思维”探索的交流智能体。载于《神经信息处理系统进展》，A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt 和 S. Levine (编)，第 36 卷，第 51991–51998 页。外部链接：文档，链接引用：§3.4。
[101] G. Li, V. Muthusamy, and H. Jacobsen (2010) A distributed service-oriented architecture for business process execution. ACM Transactions on the Web (TWEB) 4 (1), pp. 1–33. Cited by: §1. [101] G. Li, V. Muthusamy 和 H. Jacobsen (2010) 一种用于业务流程执行的分布式面向服务架构。《ACM Web 汇刊》(TWEB) 4 (1)，第 1–33 页。引用：§1。
[102] J. Li, T. Tang, W. X. Zhao, J. Nie, and J. Wen (2024) Pre-trained language models for text generation: a survey. ACM Computing Surveys 56 (9), pp. 1–39. Cited by: §1. [102] J. Li, T. Tang, W. X. Zhao, J. Nie 和 J. Wen (2024) 用于文本生成的预训练语言模型：综述。《ACM 计算调查》56 (9)，第 1–39 页。引用：§1。
[103] L. Li, Y. Fan, M. Tse, and K. Lin (2020) A review of applications in federated learning. Computers & Industrial Engineering 149, pp. 106854. Cited by: §1, §2. [103] L. Li, Y. Fan, M. Tse 和 K. Lin (2020) 联邦学习应用综述。《计算机与工业工程》149，第 106854 页。引用：§1， §2。
[104] M. Li, Y. Zhao, B. Yu, F. Song, H. Li, H. Yu, Z. Li, F. Huang, and Y. Li (2023) API-bank: a comprehensive benchmark for tool-augmented LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 3102–3116. External Links: Document Cited by: §7. [104] M. Li, Y. Zhao, B. Yu, F. Song, H. Li, H. Yu, Z. Li, F. Huang 和 Y. Li (2023) API-bank：工具增强型大语言模型的综合基准。载于《2023 年自然语言处理经验方法会议论文集》，第 3102–3116 页。外部链接：文档引用：§7。
[105] P. Li, H. Tang, T. Yang, X. Hao, T. Sang, Y. Zheng, J. Hao, M. E. Taylor, W. Tao, Z. Wang, et al. (2022) PMIC: improving multi-agent reinforcement learning with progressive mutual information collaboration. arXiv preprint arXiv:2203.08553. Cited by: §1. [105] P. Li, H. Tang, T. Yang, X. Hao, T. Sang, Y. Zheng, J. Hao, M. E. Taylor, W. Tao, Z. Wang 等 (2022) PMIC：通过渐进式互信息协作改进多智能体强化学习。 arXiv 预印本 arXiv:2203.08553。引用：§1。
[106] Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y. Li, X. Liu, and B. He (2021) A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering 35 (4), pp. 3347–3366. Cited by: §2. [106] Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y. Li, X. Liu 和 B. He (2021) 联邦学习系统综述：数据隐私与保护的愿景、炒作与现实。《IEEE 知识与数据工程汇刊》35 (4)，第 3347–3366 页。引用：§2。
[107] T. Li, S. Hu, A. Beirami, and V. Smith (2021) Ditto: fair and robust federated learning through personalization. In International conference on machine learning, pp. 6357–6368. Cited by: §2, §6.2. [107] T. Li, S. Hu, A. Beirami 和 V. Smith (2021) Ditto：通过个性化实现公平且稳健的联邦学习。载于《国际机器学习会议论文集》，第 6357–6368 页。引用：§2， §6.2。
[108] T. Li, M. Sanjabi, A. Beirami, and V. Smith (2019) Fair resource allocation in federated learning. arXiv preprint arXiv:1905.10497. Cited by: §6.2. [108] T. Li, M. Sanjabi, A. Beirami 和 V. Smith (2019) 联邦学习中的公平资源分配。 arXiv 预印本 arXiv:1905.10497。引用：§6.2。
[109] W. Li, J. Bai, and Z. Song (2026) FedAgentKE: Federated semantic knowledge evolution for heterogeneous agents. arXiv preprint arXiv:2607.21361. Cited by: §B.2, §B.6, Table 4, §C.1, §4, §4, Table 1, Table 2. [109] W. Li, J. Bai 和 Z. Song (2026) FedAgentKE：面向异构智能体的联邦语义知识演化。 arXiv 预印本 arXiv:2607.21361。引用：§B.2， §B.6，表 4， §C.1， §4， §4，表 1，表 2。
[110] H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe (2024) Let’s verify step by step. In International Conference on Learning Representations, Cited by: §B.4, §4. [110] H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever 和 K. Cobbe (2024) 让我们一步步验证。载于《国际学习表征会议论文集》，引用：§B.4， §4。
[111] T. Lin, L. Kong, S. U. Stich, and M. Jaggi (2020) Ensemble distillation for robust model fusion in federated learning. Advances in neural information processing systems 33, pp. 2351–2363. Cited by: §7. [111] T. Lin, L. Kong, S. U. Stich 和 M. Jaggi (2020) 联邦学习中用于稳健模型融合的集成蒸馏。《神经信息处理系统进展》33，第 2351–2363 页。引用：§7。
[112] Y. Lin, S. Han, H. Mao, Y. Wang, and W. J. Dally (2018) Deep gradient compression: reducing the communication bandwidth for distributed training. In Proceedings of the 6th International Conference on Learning Representations (ICLR), External Links: Link Cited by: §6.1. [112] Y. Lin, S. Han, H. Mao, Y. Wang 和 W. J. Dally (2018) 深度梯度压缩：减少分布式训练的通信带宽。载于《第 6 届国际学习表征会议 (ICLR) 论文集》，外部链接：链接引用：§6.1。
[113] Z. Lin, Y. Sun, Y. Shi, X. Wang, L. Huang, L. Shen, and D. Tao (2023) Efficient federated prompt tuning for black-box large pre-trained models. arXiv preprint arXiv:2310.03123. Cited by: §B.1, §4. [113] Z. Lin, Y. Sun, Y. Shi, X. Wang, L. Huang, L. Shen 和 D. Tao (2023) 针对黑盒大型预训练模型的高效联邦提示调优。 arXiv 预印本 arXiv:2310.03123。引用：§B.1， §4。
[114] X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang, S. Zhang, X. Deng, A. Zeng, Z. Du, C. Zhang, S. Shen, T. Zhang, Y. Su, H. Sun, M. Huang, Y. Dong, and J. Tang (2024) AgentBench: evaluating LLMs as agents. In International Conference on Learning Representations, Cited by: §6.3, §7. [114] X. Liu, H. Yu, H. Zhang, Y. Xu, X. Lei, H. Lai, Y. Gu, H. Ding, K. Men, K. Yang, S. Zhang, X. Deng, A. Zeng, Z. Du, C. Zhang, S. Shen, T. Zhang, Y. Su, H. Sun, M. Huang, Y. Dong 和 J. Tang (2024) AgentBench：评估作为智能体的大语言模型。载于《国际学习表征会议论文集》，引用：§6.3， §7。
[115] X. H. Lù, A. Kazemnejad, N. Meade, A. Patel, D. Shin, A. Zambrano, K. Stańczak, P. Shaw, C. J. Pal, and S. Reddy (2025) AgentRewardBench: Evaluating automatic evaluations of web agent trajectories. In Second Conference on Language Modeling, Cited by: §B.4, §4. [115] X. H. Lù, A. Kazemnejad, N. Meade, A. Patel, D. Shin, A. Zambrano, K. Stańczak, P. Shaw, C. J. Pal 和 S. Reddy (2025) AgentRewardBench：评估 Web 智能体轨迹的自动评估方法。载于《第二届语言建模会议论文集》，引用：§B.4， §4。
[116] Y. Lu, J. Qi, Z. Luan, S. Huang, C. Fung, H. Yang, and D. Qian (2024) FDLoRA: Personalized federated learning of large language model via dual LoRA tuning. arXiv preprint arXiv:2406.07925. Cited by: §B.1, §4. [116] Y. Lu, J. Qi, Z. Luan, S. Huang, C. Fung, H. Yang 和 D. Qian (2024) FDLoRA：通过双 LoRA 调优实现大语言模型的个性化联邦学习。 arXiv 预印本 arXiv:2406.07925。引用：§B.1， §4。
[117] X. Luo, Y. Zhang, Z. He, Z. Wang, S. Zhao, D. Li, L. K. Qiu, and Y. Yang (2025) Agent lightning: Train ANY AI agents with reinforcement learning. arXiv preprint arXiv:2508.03680. Cited by: §B.1. [117] X. Luo, Y. Zhang, Z. He, Z. Wang, S. Zhao, D. Li, L. K. Qiu 和 Y. Yang (2025) Agent lightning：使用强化学习训练任何 AI 智能体。 arXiv 预印本 arXiv:2508.03680。引用：§B.1。
[118] C. Ma, J. Li, K. Wei, B. Liu, M. Ding, L. Yuan, Z. Han, and H. V. Poor (2023) Trusted ai in multiagent systems: an overview of privacy and security for distributed learning. Proceedings of the IEEE 111 (9), pp. 1097–1132. Cited by: §2. [118] C. Ma, J. Li, K. Wei, B. Liu, M. Ding, L. Yuan, Z. Han 和 H. V. Poor (2023) 多智能体系统中的可信 AI：分布式学习的隐私与安全综述。《IEEE 会刊》111 (9)，第 1097–1132 页。引用：§2。
[119] Y. J. Ma, W. Liang, G. Wang, D. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar (2024) Eureka: Human-level reward design via coding large language models. In International Conference on Learning Representations, Cited by: §B.4, §4. [119] Y. J. Ma, W. Liang, G. Wang, D. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan 和 A. Anandkumar (2024) Eureka：通过编码大语言模型实现人类水平的奖励设计。载于《国际学习表征会议论文集》，引用：§B.4， §4。
[120] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. (2023) Self-refine: iterative refinement with self-feedback. Advances in neural information processing systems 36, pp. 46534–46594. Cited by: §7. [120] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang 等 (2023) Self-refine：带有自我反馈的迭代优化。《神经信息处理系统进展》36，第 46534–46594 页。引用：§7。
[121] Q. Mao, Q. Zhang, H. Hao, Z. Han, R. Xu, W. Jiang, Q. Hu, Z. Chen, T. Zhou, B. Li, Y. Song, J. Dong, J. Li, and P. S. Yu (2025) Privacy-preserving federated embedding learning for localized retrieval-augmented generation. arXiv preprint arXiv:2504.19101. Cited by: §B.2, Table 4, §4, Table 1. [121] Q. Mao, Q. Zhang, H. Hao, Z. Han, R. Xu, W. Jiang, Q. Hu, Z. Chen, T. Zhou, B. Li, Y. Song, J. Dong, J. Li 和 P. S. Yu (2025) 面向本地检索增强生成的隐私保护联邦嵌入学习。 arXiv 预印本 arXiv:2504.19101。引用：§B.2，表 4， §4，表 1。
[122] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Agüera y Arcas (2017) Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 54, pp. 1273–1282. Cited by: §3.4. [122] B. McMahan, E. Moore, D. Ramage, S. Hampson 和 B. Agüera y Arcas (2017) 基于去中心化数据的深度网络通信高效学习。载于《第 20 届国际人工智能与统计会议论文集》，《机器学习研究论文集》，第 54 卷，第 1273–1282 页。引用：§3.4。
[123] Y. Mei, P. Guo, M. Zhou, and V. Patel (2022) Resource-adaptive federated learning with all-in-one neural composition. In Advances in Neural Information Processing Systems, Vol. 35. Cited by: §7. [123] Y. Mei, P. Guo, M. Zhou 和 V. Patel (2022) 具有一体化神经组合的资源自适应联邦学习。载于《神经信息处理系统进展》，第 35 卷。引用：§7。
[124] K. Miettinen (1999) Nonlinear multiobjective optimization. International Series in Operations Research & Management Science, Vol. 12, Kluwer Academic Publishers, Boston. External Links: ISBN 0-7923-8278-1 Cited by: Appendix E, §3.3. [124] K. Miettinen (1999) 非线性多目标优化。《运筹学与管理科学国际丛书》，第 12 卷，Kluwer 学术出版社，波士顿。外部链接：ISBN 0-7923-8278-1 引用：附录 E， §3.3。
[125] M. Milne-Ives, C. De Cock, E. Lim, M. H. Shehadeh, N. De Pennington, G. Mole, E. Normando, and E. Meinert (2020) The effectiveness of artificial intelligence conversational agents in health care: systematic review. Journal of medical Internet research 22 (10), pp. e20346. Cited by: §1. [125] M. Milne-Ives, C. De Cock, E. Lim, M. H. Shehadeh, N. De Pennington, G. Mole, E. Normando 和 E. Meinert (2020) 医疗保健中人工智能对话智能体的有效性：系统综述。《医学互联网研究杂志》22 (10)，第 e20346 页。引用：§1。
[126] S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao (2024) Large language models: a survey. arXiv preprint arXiv:2402.06196. Cited by: §2. [126] S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain 和 J. Gao (2024) 大语言模型：综述。 arXiv 预印本 arXiv:2402.06196。引用：§2。
[127] M. Mohri, G. Sivek, and A. T. Suresh (2019) Agnostic federated learning. In International conference on machine learning, pp. 4615–4625. Cited by: §6.2. [127] M. Mohri, G. Sivek 和 A. T. Suresh (2019) 不可知联邦学习。载于《国际机器学习会议论文集》，第 4615–4625 页。引用：§6.2。
[128] J. X. Morris, V. Kuleshov, V. Shmatikov, and A. M. Rush (2023) Text embeddings reveal (almost) as much as text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, pp. 12448–12460. Cited by: §B.2, §3.3, §4. [128] J. X. Morris, V. Kuleshov, V. Shmatikov 和 A. M. Rush (2023) 文本嵌入揭示的信息几乎与文本本身一样多。载于《2023 年自然语言处理经验方法会议论文集》，新加坡，第 12448–12460 页。引用：§B.2， §3.3， §4。
[129] M. Nasr, R. Shokri, and A. Houmansadr (2019) Comprehensive privacy analysis of deep learning: passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pp. 739–753. Cited by: §6.1. [129] M. Nasr, R. Shokri 和 A. Houmansadr (2019) 深度学习的全面隐私分析：针对集中式和联邦学习的被动与主动白盒推理攻击。载于《2019 IEEE 安全与隐私研讨会 (SP)》，第 739–753 页。引用：§6.1。
[130] H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, and A. Mian (2025) A comprehensive overview of large language models. ACM Transactions on Intelligent Systems and Technology 16 (5), pp. 1–72. Cited by: §1. [130] H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes 和 A. Mian (2025) 大语言模型综合概述。《ACM 智能系统与技术汇刊》16 (5)，第 1–72 页。引用：§1。
[131] T. T. Nguyen, T. T. Huynh, Z. Ren, T. T. Nguyen, P. L. Nguyen, H. Yin, and Q. V. H. Nguyen (2025) Privacy-preserving explainable ai: a survey. Science China Information Sciences 68 (1), pp. 111101. Cited by: §2. [131] T. T. Nguyen, T. T. Huynh, Z. Ren, T. T. Nguyen, P. L. Nguyen, H. Yin 和 Q. V. H. Nguyen (2025) 隐私保护可解释 AI：综述。《中国科学：信息科学》68 (1)，第 111101 页。引用：§2。
[132] A. Norta (2015) Establishing distributed governance infrastructures for enacting cross-organization collaborations. In International Conference on Service-Oriented Computing, pp. 24–35. Cited by: §2. [132] A. Norta (2015) 建立分布式治理基础设施以实现跨组织协作。载于《国际面向服务计算会议论文集》，第 24–35 页。引用：§2。
[133] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe (2022) Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, Vol. 35, pp. 27730–27744. Cited by: §7. [133] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike 和 R. Lowe (2022) 训练语言模型通过人类反馈遵循指令。载于《神经信息处理系统进展》，第 35 卷，第 27730–27744 页。引用：§7。
[134] S. Ouyang, J. Yan, I. Hsu, Y. Chen, K. Jiang, Z. Wang, R. Han, L. T. Le, S. Daruki, X. Tang, V. Tirumalashetty, G. Lee, M. Rofouei, H. Lin, J. Han, C. Lee, and T. Pfister (2026) ReasoningBank: Scaling agent self-evolving with reasoning memory. In International Conference on Learning Representations, Cited by: §B.2, §C.1, §4. [134] S. Ouyang, J. Yan, I. Hsu, Y. Chen, K. Jiang, Z. Wang, R. Han, L. T. Le, S. Daruki, X. Tang, V. Tirumalashetty, G. Lee, M. Rofouei, H. Lin, J. Han, C. Lee 和 T. Pfister (2026) ReasoningBank：利用推理记忆扩展智能体自我演化。载于《国际学习表征会议论文集》，引用：§B.2， §C.1， §4。
[135] J. Y. Pan and J. M. Tenenbaum (1991) An intelligent agent framework for enterprise integration. IEEE Transactions on Systems, man, and cybernetics 21 (6), pp. 1391–1408. Cited by: §1. [135] J. Y. Pan 和 J. M. Tenenbaum (1991) 一种用于企业集成的智能智能体框架。《IEEE 系统、人与控制论汇刊》21 (6)，第 1391–1408 页。引用：§1。
[136] J. Pan, Y. Zhang, N. Tomlin, Y. Zhou, S. Levine, and A. Suhr (2024) Autonomous evaluation and refinement of digital agents. In First Conference on Language Modeling, Cited by: §B.4. [136] J. Pan, Y. Zhang, N. Tomlin, Y. Zhou, S. Levine 和 A. Suhr (2024) 数字智能体的自动评估与优化。载于《第一届语言建模会议论文集》，引用：§B.4。
[137] S. G. Patil, T. Zhang, X. Wang, and J. E. Gonzalez (2024) Gorilla: Large language model connected with massive APIs. In Advances in Neural Information Processing Systems, Vol. 37, pp. 126544–126565. Cited by: §B.3. [137] S. G. Patil, T. Zhang, X. Wang 和 J. E. Gonzalez (2024) Gorilla：连接海量 API 的大语言模型。载于《神经信息处理系统进展》，第 37 卷，第 126544–126565 页。引用：§B.3。
[138] H. Peng, H. Li, Y. Song, V. W. Zheng, and J. Li (2021) Differentially private federated knowledge graphs embedding. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 1416–1425. Cited by: §B.5, §C.1, Appendix D, §4, Table 1, Table 2. [138] H. Peng, H. Li, Y. Song, V. W. Zheng 和 J. Li (2021) 差分隐私联邦知识图谱嵌入。载于《第 30 届 ACM 国际信息与知识管理会议论文集》，第 1416–1425 页。引用：§B.5， §C.1，附录 D， §4，表 1，表 2。
[139] I. Petrov, D. I. Dimitrov, M. Baader, M. N. Müller, and M. Vechev (2024) DAGER: Exact gradient inversion for large language models. In Advances in Neural Information Processing Systems, Vol. 37, pp. 87801–87830. Cited by: §B.1, §3.3, §4. [139] I. Petrov, D. I. Dimitrov, M. Baader, M. N. Müller 和 M. Vechev (2024) DAGER：大语言模型的精确梯度反演。载于《神经信息处理系统进展》，第 37 卷，第 87801–87830 页。引用：§B.1， §3.3， §4。
[140] E. Pignatelli, J. Ferret, M. Geist, T. Mesnard, H. van Hasselt, and L. Toni (2024) A survey of temporal credit assignment in deep reinforcement learning. Transactions on Machine Learning Research 2024. Note: Survey Certification External Links: ISSN 2835-8856 Cited by: §B.4. [140] E. Pignatelli, J. Ferret, M. Geist, T. Mesnard, H. van Hasselt 和 L. Toni (2024) 深度强化学习中时间信用分配综述。《机器学习研究汇刊》2024。注：综述认证外部链接：ISSN 2835-8856 引用：§B.4。
[141] P. Ponnusamy, A. Ghias, Y. Yi, B. Yao, C. Guo, and R. Sarikaya (2022) Feedback-based self-learning in large-scale conversational ai agents. AI magazine 42 (4), pp. 43–56. Cited by: §1. [141] P. Ponnusamy, A. Ghias, Y. Yi, B. Yao, C. Guo 和 R. Sarikaya (2022) 大规模对话 AI 智能体中的基于反馈的自学习。《AI 杂志》42 (4)，第 43–56 页。引用：§1。
[142] S. M. Preum, S. Munir, M. Ma, M. S. Yasar, D. J. Stone, R. Williams, H. Alemzadeh, and J. A. Stankovic (2021) A review of cognitive assistants for healthcare: trends, prospects, and future directions. ACM Computing Surveys (CSUR) 53 (6), pp. 1–37. Cited by: §1. [142] S. M. Preum, S. Munir, M. Ma, M. S. Yasar, D. J. Stone, R. Williams, H. Alemzadeh 和 J. A. Stankovic (2021) 医疗保健认知助手综述：趋势、前景与未来方向。《ACM 计算调查》(CSUR) 53 (6)，第 1–37 页。引用：§1。
[143] P. Putta, E. Mills, N. Garg, S. Motwani, C. Finn, D. Garg, and R. Rafailov (2024) Agent q: advanced reasoning and learning for autonomous ai agents. arXiv preprint arXiv:2408.07199. Cited by: §1, §1. [143] P. Putta, E. Mills, N. Garg, S. Motwani, C. Finn, D. Garg 和 R. Rafailov (2024) Agent q：用于自主 AI 智能体的高级推理与学习。 arXiv 预印本 arXiv:2408.07199。引用：§1， §1。
[144] J. Qi, Q. Zhou, L. Lei, and K. Zheng (2021) Federated reinforcement learning: Techniques, applications, and open challenges. Intelligence & Robotics 1 (1), pp. 18–57. Cited by: §B.1. [144] J. Qi, Q. Zhou, L. Lei 和 K. Zheng (2021) 联邦强化学习：技术、应用与开放挑战。《智能与机器人》1 (1)，第 18–57 页。引用：§B.1。
[145] P. Qi, D. Chiaro, A. Guzzo, M. Ianni, G. Fortino, and F. Piccialli (2024) Model aggregation techniques in federated learning: a comprehensive survey. Future Generation Computer Systems 150, pp. 272–293. Cited by: §2. [145] P. Qi, D. Chiaro, A. Guzzo, M. Ianni, G. Fortino 和 F. Piccialli (2024) 联邦学习中的模型聚合技术：综合综述。《未来一代计算机系统》150，第 272–293 页。引用：§2。
[146] C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong, et al. (2024) Chatdev: communicative agents for software development. In Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers), pp. 15174–15186. Cited by: §1. [146] C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong 等 (2024) Chatdev：用于软件开发的交流智能体。载于《第 62 届计算语言学协会年会论文集（第一卷：长论文）》，第 15174–15186 页。引用：§1。
[147] C. Qian, C. Han, Y. R. Fung, Y. Qin, Z. Liu, and H. Ji (2023) CREATOR: Tool creation for disentangling abstract and concrete reasoning of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, pp. 6922–6939. Cited by: §B.3. [147] C. Qian, C. Han, Y. R. Fung, Y. Qin, Z. Liu 和 H. Ji (2023) CREATOR：用于解构大语言模型抽象与具体推理的工具创建。载于《计算语言学协会研究结果：EMNLP 2023》，新加坡，第 6922–6939 页。引用：§B.3。
[148] Z. Qiang, W. Wang, and K. Taylor (2024) Agent-OM: Leveraging LLM agents for ontology matching. Proceedings of the VLDB Endowment 18 (3), pp. 516–529. Cited by: §B.5, §C.1, §4. [148] Z. Qiang, W. Wang 和 K. Taylor (2024) Agent-OM：利用 LLM 智能体进行本体匹配。《VLDB 基金会论文集》18 (3)，第 516–529 页。引用：§B.5， §C.1， §4。
[149] Y. Qin, S. Hu, Y. Lin, W. Chen, N. Ding, G. Cui, Z. Zeng, X. Zhou, Y. Huang, C. Xiao, C. Han, Y. R. Fung, Y. Su, H. Wang, C. Qian, R. Tian, K. Zhu, S. Liang, X. Shen, B. Xu, Z. Zhang, Y. Ye, B. Li, Z. Tang, J. Yi, Y. Zhu, Z. Dai, L. Yan, X. Cong, Y. Lu, W. Zhao, Y. Huang, J. Yan, X. Han, X. Sun, D. Li, J. Phang, C. Yang, T. Wu, H. Ji, G. Li, Z. Liu, and M. Sun (2024) Tool learning with foundation models. ACM Computing Surveys 57 (4), pp. 102:1–102:37. Cited by: §B.3. [149] Y. Qin, S. Hu, Y. Lin, W. Chen, N. Ding, G. Cui, Z. Zeng, X. Zhou, Y. Huang, C. Xiao, C. Han, Y. R. Fung, Y. Su, H. Wang, C. Qian, R. Tian, K. Zhu, S. Liang, X. Shen, B. Xu, Z. Zhang, Y. Ye, B. Li, Z. Tang, J. Yi, Y. Zhu, Z. Dai, L. Yan, X. Cong, Y. Lu, W. Zhao, Y. Huang, J. Yan, X. Han, X. Sun, D. Li, J. Phang, C. Yang, T. Wu, H. Ji, G. Li, Z. Liu 和 M. Sun (2024) 基础模型的工具学习。《ACM 计算调查》57 (4)，第 102:1–102:37 页。引用：§B.3。
[150] Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, et al. (2024) Toolllm: facilitating large language models to master 16000+ real-world apis. In International Conference on Learning Representations, Vol. 2024, pp. 9695–9717. Cited by: §B.3, §4. [150] Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian 等 (2024) Toolllm：助力大语言模型掌握 16000+ 现实世界 API。载于《国际学习表征会议论文集》，第 2024 卷，第 9695–9717 页。引用：§B.3， §4。
[151] Z. Qin, D. Chen, B. Qian, B. Ding, Y. Li, and S. Deng (2024) Federated full-parameter tuning of billion-sized language models with communication cost under 18 kilobytes. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, pp. 41473–41497. Cited by: §B.1, §B.6, §4. [151] Z. Qin, D. Chen, B. Qian, B. Ding, Y. Li 和 S. Deng (2024) 十亿级语言模型的联邦全参数调优，通信成本低于 18 千字节。载于《第 41 届国际机器学习会议论文集》，《机器学习研究论文集》，第 235 卷，第 41473–41497 页。引用：§B.1， §B.6， §4。
[152] C. Qu, S. Dai, X. Wei, H. Cai, S. Wang, D. Yin, J. Xu, and J. Wen (2025) From exploration to mastery: Enabling LLMs to master tools via self-driven interactions. In International Conference on Learning Representations, Cited by: §B.3, §4. [152] C. Qu, S. Dai, X. Wei, H. Cai, S. Wang, D. Yin, J. Xu 和 J. Wen (2025) 从探索到精通：通过自驱动交互使 LLM 掌握工具。载于《国际学习表征会议论文集》，引用：§B.3， §4。
[153] C. Qu, S. Dai, X. Wei, H. Cai, S. Wang, D. Yin, J. Xu, and J. Wen (2025) Tool learning with large language models: A survey. Frontiers of Computer Science 19 (8), pp. 198343. Cited by: §B.3. [153] C. Qu, S. Dai, X. Wei, H. Cai, S. Wang, D. Yin, J. Xu 和 J. Wen (2025) 大语言模型的工具学习：综述。《计算机科学前沿》19 (8)，第 198343 页。引用：§B.3。
[154] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn (2023) Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems, Vol. 36, pp. 53728–53741. Cited by: §7. [154] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon 和 C. Finn (2023) 直接偏好优化：你的语言模型暗中是一个奖励模型。载于《神经信息处理系统进展》，第 36 卷，第 53728–53741 页。引用：§7。
[155] Z. Rasheed, M. Waseem, A. Ahmad, K. Kemell, W. Xiaofeng, A. N. Duc, and P. Abrahamsson (2024) Can large language models serve as data analysts? a multi-agent assisted approach for qualitative data analysis. arXiv preprint arXiv:2402.01386. Cited by: §1. [155] Z. Rasheed, M. Waseem, A. Ahmad, K. Kemell, W. Xiaofeng, A. N. Duc 和 P. Abrahamsson (2024) 大语言模型能担任数据分析师吗？一种用于定性数据分析的多智能体辅助方法。 arXiv 预印本 arXiv:2402.01386。引用：§1。
[156] B. Ray Chaudhury, L. Li, M. Kang, B. Li, and R. Mehta (2022) Fairness in federated learning via core-stability. Advances in neural information processing systems 35, pp. 5738–5750. Cited by: §7. [156] B. Ray Chaudhury, L. Li, M. Kang, B. Li 和 R. Mehta (2022) 通过核心稳定性实现联邦学习中的公平性。《神经信息处理系统进展》35，第 5738–5750 页。引用：§7。
[157] A. Rezazadeh, Z. Li, A. Lou, Y. Zhao, W. Wei, and Y. Bao (2025) Collaborative memory: Multi-user memory sharing in LLM agents with dynamic access control. arXiv preprint arXiv:2505.18279. Cited by: §B.2, §4. [157] A. Rezazadeh, Z. Li, A. Lou, Y. Zhao, W. Wei 和 Y. Bao (2025) 协同记忆：具有动态访问控制的 LLM 智能体中的多用户记忆共享。 arXiv 预印本 arXiv:2505.18279。引用：§B.2， §4。
[158] M. Rizinski and D. Trajanov (2025) Ai agents in finance and fintech: a scientific review of agent-based systems, applications, and future horizons. Computers, Materials & Continua 86 (1), pp. 1–34. Cited by: §1. [158] M. Rizinski 和 D. Trajanov (2025) 金融与金融科技中的 AI 智能体：基于智能体的系统、应用与未来视野的科学综述。《计算机、材料与连续体》86 (1)，第 1–34 页。引用：§1。
[159] Y. Rizk, M. Awad, and E. W. Tunstel (2018) Decision making in multiagent systems: a survey. IEEE Transactions on Cognitive and Developmental Systems 10 (3), pp. 514–529. Cited by: §1. [159] Y. Rizk, M. Awad 和 E. W. Tunstel (2018) 多智能体系统中的决策：综述。《IEEE 认知与发展系统汇刊》10 (3)，第 514–529 页。引用：§1。
[160] C. Roda, A. Angehrn, T. Nabeth, and L. Razmerita (2003) Using conversational agents to support the adoption of knowledge sharing practices. Interacting with computers 15 (1), pp. 57–89. Cited by: §1. [160] C. Roda, A. Angehrn, T. Nabeth 和 L. Razmerita (2003) 利用对话智能体支持知识共享实践的采用。《与计算机交互》15 (1)，第 57–89 页。引用：§1。
[161] R. Saha, G. Kumar, G. Geetha, M. Conti, and W. J. Buchanan (2024) Application of randomness for security and privacy in multi-party computation. IEEE Transactions on Dependable and Secure Computing 21 (6), pp. 5694–5705. Cited by: §2. [161] R. Saha, G. Kumar, G. Geetha, M. Conti 和 W. J. Buchanan (2024) 随机性在多方计算安全与隐私中的应用。《IEEE 可信与安全计算汇刊》21 (6)，第 5694–5705 页。引用：§2。
[162] H. Salman, C. Zaki, N. Charara, S. Guehis, J. Pradat-Peyre, and A. Nasser (2025) Knowledge distillation in federated learning: a comprehensive survey. Discover Computing 28 (1), pp. 145. Cited by: §2. [162] H. Salman, C. Zaki, N. Charara, S. Guehis, J. Pradat-Peyre 和 A. Nasser (2025) 联邦学习中的知识蒸馏：综合综述。《发现计算》28 (1)，第 145 页。引用：§2。
[163] H. U. Sami, S. Sen, A. K. Roy-Chowdhury, S. V. Krishnamurthy, and B. Guler (2025) Gradient inversion attacks on parameter-efficient fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10224–10234. Cited by: §B.1, §4. [163] H. U. Sami, S. Sen, A. K. Roy-Chowdhury, S. V. Krishnamurthy 和 B. Guler (2025) 针对参数高效微调的梯度反演攻击。载于《IEEE/CVF 计算机视觉与模式识别会议论文集》，第 10224–10234 页。引用：§B.1， §4。
[164] F. Sattler, K. Müller, and W. Samek (2021) Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems 32 (8), pp. 3710–3722. Cited by: §5.3. [164] F. Sattler, K. Müller 和 W. Samek (2021) 聚类联邦学习：隐私约束下的模型不可知分布式多任务优化。《IEEE 神经网络与学习系统汇刊》32 (8)，第 3710–3722 页。引用：§5.3。
[165] T. Schick, J. Dwivedi-Yu, R. Dessi, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom (2023) Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, Vol. 36, pp. 68539–68551. External Links: Document Cited by: §B.3, §4. [165] T. Schick, J. Dwivedi-Yu, R. Dessi, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda 和 T. Scialom (2023) Toolformer：语言模型可以自学使用工具。载于《神经信息处理系统进展》，第 36 卷，第 68539–68551 页。外部链接：文档引用：§B.3， §4。
[166] Y. Shao, T. Li, W. Shi, Y. Liu, and D. Yang (2024) PrivacyLens: Evaluating privacy norm awareness of language models in action. In Advances in Neural Information Processing Systems, Vol. 37, pp. 89373–89407. Cited by: §3.3. [166] Y. Shao, T. Li, W. Shi, Y. Liu 和 D. Yang (2024) PrivacyLens：评估语言模型在行动中的隐私规范意识。载于《神经信息处理系统进展》，第 37 卷，第 89373–89407 页。引用：§3.3。
[167] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao (2023) Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, Vol. 36, pp. 8634–8652. Cited by: §B.2, §C.1, §3.1, §4. [167] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan 和 S. Yao (2023) Reflexion：带有语言强化学习的语言智能体。载于《神经信息处理系统进展》，第 36 卷，第 8634–8652 页。引用：§B.2， §C.1， §3.1， §4。
[168] M. Srewa, T. Zhao, and S. Elmalaki (2025) PluralLLM: Pluralistic alignment in LLMs via federated learning. In Proceedings of the 3rd International Workshop on Human-Centered Sensing, Modeling, and Intelligent Systems, Irvine, CA, USA, pp. 64–69. Cited by: §B.4, §4, Table 1. [168] M. Srewa, T. Zhao 和 S. Elmalaki (2025) PluralLLM：通过联邦学习实现 LLM 中的多元化对齐。载于《第三届以人为本的传感、建模与智能系统国际研讨会论文集》，美国加利福尼亚州欧文市，第 64–69 页。引用：§B.4， §4，表 1。
[169] T. R. Sumers, S. Yao, K. Narasimhan, and T. L. Griffiths (2024) Cognitive architectures for language agents. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: §3.1. [169] T. R. Sumers, S. Yao, K. Narasimhan 和 T. L. Griffiths (2024) 语言智能体的认知架构。《机器学习研究汇刊》。外部链接：ISSN 2835-8856 引用：§3.1。
[170] J. Sun, Z. Xu, H. Yin, D. Yang, D. Xu, Y. Liu, Z. Du, Y. Chen, and H. R. Roth (2024) FedBPT: Efficient federated black-box prompt tuning for large language models. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, pp. 47159–47173. Cited by: §B.1, §4, Table 1. [170] J. Sun, Z. Xu, H. Yin, D. Yang, D. Xu, Y. Liu, Z. Du, Y. Chen 和 H. R. Roth (2024) FedBPT：针对大语言模型的高效联邦黑盒提示调优。载于《第 41 届国际机器学习会议论文集》，《机器学习研究论文集》，第 235 卷，第 47159–47173 页。引用：§B.1， §4，表 1。
[171] Y. Sun, Z. Li, Y. Li, and B. Ding (2024) Improving lora in privacy-preserving federated learning. In International conference on learning representations, Vol. 2024, pp. 17978–17994. Cited by: §B.1, §2, §4. [171] Y. Sun, Z. Li, Y. Li 和 B. Ding (2024) 在隐私保护联邦学习中改进 LoRA。载于《国际学习表征会议论文集》，第 2024 卷，第 17978–17994 页。引用：§B.1， §2， §4。
[172] C. T Dinh, N. Tran, and J. Nguyen (2020) Personalized federated learning with moreau envelopes. Advances in neural information processing systems 33, pp. 21394–21405. Cited by: §6.2. [172] C. T Dinh, N. Tran 和 J. Nguyen (2020) 带有 Moreau 包络的个性化联邦学习。《神经信息处理系统进展》33，第 21394–21405 页。引用：§6.2。
[173] A. Z. Tan, H. Yu, L. Cui, and Q. Yang (2022) Towards personalized federated learning. IEEE transactions on neural networks and learning systems 34 (12), pp. 9587–9603. Cited by: §2. [173] A. Z. Tan, H. Yu, L. Cui 和 Q. Yang (2022) 迈向个性化联邦学习。《IEEE 神经网络与学习系统汇刊》34 (12)，第 9587–9603 页。引用：§2。
[174] X. Tang, T. Qin, T. Peng, Z. Zhou, D. Shao, T. Du, X. Wei, P. Xia, F. Wu, H. Zhu, G. Zhang, J. Liu, X. Wang, S. Hong, C. Wu, H. Cheng, C. Wang, and W. Zhou (2025) Agent KB: Leveraging cross-domain experience for agentic problem solving. arXiv preprint arXiv:2507.06229. Cited by: §B.2, §3.4, §4, §5.3. [174] X. Tang, T. Qin, T. Peng, Z. Zhou, D. Shao, T. Du, X. Wei, P. Xia, F. Wu, H. Zhu, G. Zhang, J. Liu, X. Wang, S. Hong, C. Wu, H. Cheng, C. Wang 和 W. Zhou (2025) Agent KB：利用跨领域经验进行智能体问题解决。 arXiv 预印本 arXiv:2507.06229。引用：§B.2， §3.4， §4， §5.3。
[175] X. Tang, R. Shin, H. A. Inan, A. Manoel, F. Mireshghallah, Z. Lin, S. Gopi, J. Kulkarni, and R. Sim (2024) Privacy-preserving in-context learning with differentially private few-shot generation. In International Conference on Learning Representations, Cited by: §B.2, §4. [175] X. Tang, R. Shin, H. A. Inan, A. Manoel, F. Mireshghallah, Z. Lin, S. Gopi, J. Kulkarni 和 R. Sim (2024) 带有差分隐私少样本生成的隐私保护上下文学习。载于《国际学习表征会议论文集》，引用：§B.2， §4。
[176] Q. Tian, K. Kuang, F. Liu, and B. Wang (2023) Learning from good trajectories in offline multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, pp. 11672–11680. Cited by: §2. [176] Q. Tian, K. Kuang, F. Liu 和 B. Wang (2023) 在离线多智能体强化学习中从良好轨迹中学习。载于《AAAI 人工智能会议论文集》，第 37 卷，第 11672–11680 页。引用：§2。
[177] L. Van Elst, V. Dignum, and A. Abecker (2003) Towards agent-mediated knowledge management. In International Symposium on Agent-Mediated Knowledge Management, pp. 1–30. Cited by: §1. [177] L. Van Elst, V. Dignum 和 A. Abecker (2003) 迈向智能体中介的知识管理。载于《智能体中介知识管理国际研讨会论文集》，第 1–30 页。引用：§1。
[178] T. Verhagen, J. Van Nes, F. Feldberg, and W. Van Dolen (2014) Virtual customer service agents: using social presence and personalization to shape online service encounters. Journal of Computer-Mediated Communication 19 (3), pp. 529–545. Cited by: §1. [178] T. Verhagen, J. Van Nes, F. Feldberg 和 W. Van Dolen (2014) 虚拟客户服务智能体：使用社会临场感和个性化塑造在线服务接触。《计算机媒介传播杂志》19 (3)，第 529–545 页。引用：§1。
[179] B. Wang, W. He, S. Zeng, Z. Xiang, Y. Xing, J. Tang, and P. He (2025) Unveiling privacy risks in LLM agent memory. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vienna, Austria, pp. 25241–25260. Cited by: §B.2, §3.3, §4. [179] B. Wang, W. He, S. Zeng, Z. Xiang, Y. Xing, J. Tang 和 P. He (2025) 揭示 LLM 智能体记忆中的隐私风险。载于《第 63 届计算语言学协会年会论文集（第一卷：长论文）》，奥地利维也纳，第 25241–25260 页。引用：§B.2， §3.3， §4。
[180] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar (2024) Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: §B.5, §C.1. [180] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan 和 A. Anandkumar (2024) Voyager：一种具有大语言模型的开放式具身智能体。《机器学习研究汇刊》。外部链接：ISSN 2835-8856 引用：§B.5， §C.1。
[181] J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V. Poor (2020) Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems 33, pp. 7611–7623. Cited by: §6.1. [181] J. Wang, Q. Liu, H. Liang, G. Joshi 和 H. V. Poor (2020) 解决异构联邦优化中的目标不一致问题。《神经信息处理系统进展》33，第 7611–7623 页。引用：§6.1。
[182] L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin, et al. (2024) A survey on large language model based autonomous agents. Frontiers of computer science 18 (6), pp. 186345. Cited by: §1. [182] L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin 等 (2024) 基于大语言模型的自主智能体综述。《计算机科学前沿》18 (6)，第 186345 页。引用：§1。
[183] S. Wang, E. Khramtsova, S. Zhuang, and G. Zuccon (2024) FeB4RAG: Evaluating federated search in the context of retrieval augmented generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 763–773. Cited by: Table 1. [183] S. Wang, E. Khramtsova, S. Zhuang 和 G. Zuccon (2024) FeB4RAG：在检索增强生成背景下评估联邦搜索。载于《第 47 届国际 ACM SIGIR 信息检索研究与开发会议论文集》，第 763–773 页。引用：表 1。
[184] T. Wang, J. Rausch, C. Zhang, R. Jia, and D. Song (2020) A principled approach to data valuation for federated learning. In Federated Learning: Privacy and Incentive, Lecture Notes in Computer Science, Vol. 12500, pp. 153–167. Cited by: §5.3. [184] T. Wang, J. Rausch, C. Zhang, R. Jia 和 D. Song (2020) 一种用于联邦学习数据估值的原则性方法。载于《联邦学习：隐私与激励》，《计算机科学讲义》，第 12500 卷，第 153–167 页。引用：§5.3。
[185] W. Wang, H. Shi, M. Yuan, Y. Lin, P. Tong, H. Zhou, G. Liu, P. Zhao, Y. Wang, and S. Chen (2026) FedGUI: Benchmarking federated GUI agents across heterogeneous platforms, devices, and operating systems. In Findings of the Association for Computational Linguistics: ACL 2026, San Diego, California, United States, pp. 28747–28767. Cited by: §B.1, Table 4, §4, Table 1. [185] W. Wang, H. Shi, M. Yuan, Y. Lin, P. Tong, H. Zhou, G. Liu, P. Zhao, Y. Wang 和 S. Chen (2026) FedGUI：跨异构平台、设备和操作系统对联邦 GUI 智能体进行基准测试。载于《计算语言学协会研究结果：ACL 2026》，美国加利福尼亚州圣地亚哥，第 28747–28767 页。引用：§B.1，表 4， §4，表 1。
[186] W. Wang, Z. Yu, R. Ye, J. Zhang, G. Liu, L. Liu, S. Chen, and Y. Wang (2025) FedMABench: Benchmarking mobile GUI agents on decentralized heterogeneous user data. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China, pp. 26387–26408. Cited by: §B.1, Table 4, §4, Table 1. [186] W. Wang, Z. Yu, R. Ye, J. Zhang, G. Liu, L. Liu, S. Chen 和 Y. Wang (2025) FedMABench：在去中心化异构用户数据上对移动 GUI 智能体进行基准测试。载于《2025 年自然语言处理经验方法会议论文集》，中国苏州，第 26387–26408 页。引用：§B.1，表 4， §4，表 1。
[187] W. Wang, M. Yuan, Z. Yu, G. Liu, R. Ye, T. Jin, S. Chen, and Y. Wang (2025) MobileA3gent: Training mobile GUI agents using decentralized self-sourced data from diverse users. In Proceedings of the Fourth Workshop on Bridging Human-Computer Interaction and Natural Language Processing, Suzhou, China, pp. 79–112. Note: Earlier arXiv version (arXiv:2502.02982v1) titled “FedMobileAgent” Cited by: §B.1, Table 4, §4, Table 1. [187] W. Wang, M. Yuan, Z. Yu, G. Liu, R. Ye, T. Jin, S. Chen 和 Y. Wang (2025) MobileA3gent：使用来自不同用户的去中心化自源数据训练移动 GUI 智能体。载于《第四届人机交互与自然语言处理桥接研讨会论文集》，中国苏州，第 79–112 页。注：较早的 arXiv 版本 (arXiv:2502.02982v1) 标题为“FedMobileAgent” 引用：§B.1，表 4， §4，表 1。
[188] Y. Wang, J. Zhang, Z. Wu, T. Cai, Z. Liu, Z. Sun, M. Dong, M. Zheng, Y. Duan, X. Yin, et al. (2026) From agent traces to trust: a survey of evidence tracing and execution provenance in llm agents. arXiv preprint arXiv:2606.04990. Cited by: §1. [188] Y. Wang, J. Zhang, Z. Wu, T. Cai, Z. Liu, Z. Sun, M. Dong, M. Zheng, Y. Duan, X. Yin 等 (2026) 从智能体痕迹到信任：LLM 智能体中证据追踪与执行来源综述。 arXiv 预印本 arXiv:2606.04990。引用：§1。
[189] Z. Wang, G. Neubig, and D. Fried (2024) TroVE: Inducing verifiable and efficient toolboxes for solving programmatic tasks. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, pp. 51177–51191. Cited by: §B.3. [189] Z. Wang, G. Neubig 和 D. Fried (2024) TroVE：诱导可验证且高效的工具箱以解决程序化任务。载于《第 41 届国际机器学习会议论文集》，《机器学习研究论文集》，第 235 卷，第 51177–51191 页。引用：§B.3。
[190] Z. Wang, S. Cai, G. Chen, A. Liu, X. S. Ma, and Y. Liang (2023) Describe, explain, plan and select: interactive planning with llms enables open-world multi-task agents. Advances in Neural Information Processing Systems 36, pp. 34153–34189. Cited by: §1, §1. [190] Z. Wang, S. Cai, G. Chen, A. Liu, X. S. Ma 和 Y. Liang (2023) 描述、解释、规划与选择：与 LLM 的交互式规划支持开放世界多任务智能体。《神经信息处理系统进展》36，第 34153–34189 页。引用：§1， §1。
[191] Z. Wang, Y. Cui, L. Zhong, Z. Zhang, D. Yin, B. Y. Lin, and J. Shang (2024) Officebench: benchmarking language agents across multiple applications for office automation. arXiv preprint arXiv:2407.19056. Cited by: §1. [191] Z. Wang, Y. Cui, L. Zhong, Z. Zhang, D. Yin, B. Y. Lin 和 J. Shang (2024) Officebench：跨多个办公自动化应用程序对语言智能体进行基准测试。 arXiv 预印本 arXiv:2407.19056。引用：§1。
[192] Z. Wang, Z. Dai, B. Póczos, and J. Carbonell (2019) Characterizing and avoiding negative transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11285–11294. Cited by: §5.3. [192] Z. Wang, Z. Dai, B. Póczos 和 J. Carbonell (2019) 表征并避免负迁移。载于《IEEE/CVF 计算机视觉与模式识别会议论文集》，第 11285–11294 页。引用：§5.3。
[193] Z. Wang, Z. Shen, Y. He, G. Sun, H. Wang, L. Lyu, and A. Li (2024) FLoRA: federated fine-tuning large language models with heterogeneous low-rank adaptations. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, pp. 22513–22533. External Links: Document, Link Cited by: §B.1, §2, §4. [193] Z. Wang, Z. Shen, Y. He, G. Sun, H. Wang, L. Lyu 和 A. Li (2024) FLoRA：具有异构低秩适应的大语言模型联邦微调。载于《神经信息处理系统进展》，A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak 和 C. Zhang (编)，第 37 卷，第 22513–22533 页。外部链接：文档，链接引用：§B.1， §2， §4。
[194] Z. Z. Wang, J. Mao, D. Fried, and G. Neubig (2025) Agent workflow memory. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, pp. 63897–63911. Cited by: §B.2, §C.1, §4. [194] Z. Z. Wang, J. Mao, D. Fried 和 G. Neubig (2025) 智能体工作流记忆。载于《第 42 届国际机器学习会议论文集》，《机器学习研究论文集》，第 267 卷，第 63897–63911 页。引用：§B.2， §C.1， §4。
[195] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. Quek, and H. V. Poor (2020) Federated learning with differential privacy: algorithms and performance analysis. IEEE transactions on information forensics and security 15, pp. 3454–3469. Cited by: §2. [195] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. Quek 和 H. V. Poor (2020) 具有差分隐私的联邦学习：算法与性能分析。《IEEE 信息取证与安全汇刊》15，第 3454–3469 页。引用：§2。
[196] J. Wen, Z. Zhang, Y. Lan, Z. Cui, J. Cai, and W. Zhang (2023) A survey on federated learning: challenges and applications. International journal of machine learning and cybernetics 14 (2), pp. 513–535. Cited by: §1, §2. [196] J. Wen, Z. Zhang, Y. Lan, Z. Cui, J. Cai 和 W. Zhang (2023) 联邦学习综述：挑战与应用。《国际机器学习与控制论杂志》14 (2)，第 513–535 页。引用：§1， §2。
[197] D. Wu (2001) Software agents for knowledge management: coordination in multi-agent supply chains and auctions. Expert Systems with Applications 20 (1), pp. 51–64. Cited by: §1. [197] D. Wu (2001) 用于知识管理的软件智能体：多智能体供应链与拍卖中的协调。《专家系统及其应用》20 (1)，第 51–64 页。引用：§1。
[198] F. Wu, Z. Li, Y. Li, B. Ding, and J. Gao (2024) FedBiOT: LLM local fine-tuning in federated learning without full model. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3345–3355. Cited by: §2. [198] F. Wu, Z. Li, Y. Li, B. Ding 和 J. Gao (2024) FedBiOT：联邦学习中无需完整模型的 LLM 本地微调。载于《第 30 届 ACM SIGKDD 知识发现与数据挖掘会议论文集》，第 3345–3355 页。引用：§2。
[199] F. Wu, X. Liu, H. Wang, X. Wang, L. Su, and J. Gao (2025) Towards federated RLHF with aggregated client preference for LLMs. In International Conference on Learning Representations, Cited by: §B.4, §B.4, Table 4, §C.1, §4, Table 1, Table 2. [199] F. Wu, X. Liu, H. Wang, X. Wang, L. Su 和 J. Gao (2025) 迈向具有聚合客户偏好的 LLM 联邦 RLHF。载于《国际学习表征会议论文集》，引用：§B.4， §B.4，表 4， §C.1， §4，表 1，表 2。
[200] P. Wu, K. Li, J. Nan, and F. Wang (2024) Federated in-context LLM agent learning. arXiv preprint arXiv:2412.08054. Cited by: §B.3, §B.3, §B.6, Table 4, §3.4, §4, §4, Table 1. [200] P. Wu, K. Li, J. Nan 和 F. Wang (2024) 联邦上下文 LLM 智能体学习。 arXiv 预印本 arXiv:2412.08054。引用：§B.3， §B.3， §B.6，表 4， §3.4， §4， §4，表 1。
[201] Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, A. H. Awadallah, R. W. White, D. Burger, and C. Wang (2023) AutoGen: enabling next-gen llm applications via multi-agent conversation. arXiv. External Links: 2308.08155 Cited by: §3.4. [201] Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, A. H. Awadallah, R. W. White, D. Burger 和 C. Wang (2023) AutoGen：通过多智能体对话实现下一代 LLM 应用。 arXiv。外部链接：2308.08155 引用：§3.4。
[202] T. Wu, A. Panda, J. T. Wang, and P. Mittal (2024) Privacy-preserving in-context learning for large language models. In International Conference on Learning Representations, Cited by: §B.2, §4. [202] T. Wu, A. Panda, J. T. Wang 和 P. Mittal (2024) 大语言模型的隐私保护上下文学习。载于《国际学习表征会议论文集》，引用：§B.2， §4。
[203] Y. Wu, C. Tian, J. Li, H. Sun, K. Tam, Z. Zhou, H. Liao, J. Xiong, Z. Guo, L. Li, and C. Xu (2026) A survey on federated fine-tuning of large language models. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: §B.1, §4. [203] Y. Wu, C. Tian, J. Li, H. Sun, K. Tam, Z. Zhou, H. Liao, J. Xiong, Z. Guo, L. Li 和 C. Xu (2026) 大语言模型联邦微调综述。《机器学习研究汇刊》。外部链接：ISSN 2835-8856 引用：§B.1， §4。
[204] C. Xie, Z. Lin, A. Backurs, S. Gopi, D. Yu, H. A. Inan, H. Nori, H. Jiang, H. Zhang, Y. T. Lee, B. Li, and S. Yekhanin (2024) Differentially private synthetic data via foundation model APIs 2: Text. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, pp. 54531–54560. Cited by: §B.2, §4. [204] C. Xie, Z. Lin, A. Backurs, S. Gopi, D. Yu, H. A. Inan, H. Nori, H. Jiang, H. Zhang, Y. T. Lee, B. Li 和 S. Yekhanin (2024) 通过基础模型 API 的差分隐私合成数据 2：文本。载于《第 41 届国际机器学习会议论文集》，《机器学习研究论文集》，第 235 卷，第 54531–54560 页。引用：§B.2， §4。
[205] T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y. Liu, Y. Xu, S. Zhou, S. Savarese, C. Xiong, V. Zhong, and T. Yu (2024) OSWorld: benchmarking multimodal agents for open-ended tasks in real computer environments. In Advances in Neural Information Processing Systems, Vol. 37. Cited by: §6.3, §7. [205] T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y. Liu, Y. Xu, S. Zhou, S. Savarese, C. Xiong, V. Zhong 和 T. Yu (2024) OSWorld：在真实计算机环境中为开放式任务进行多模态智能体基准测试。载于《神经信息处理系统进展》，第 37 卷。引用：§6.3， §7。
[206] T. Xie, S. Zhao, C. H. Wu, Y. Liu, Q. Luo, V. Zhong, Y. Yang, and T. Yu (2024) Text2Reward: Reward shaping with language models for reinforcement learning. In International Conference on Learning Representations, Cited by: §B.4, §4. [206] T. Xie, S. Zhao, C. H. Wu, Y. Liu, Q. Luo, V. Zhong, Y. Yang 和 T. Yu (2024) Text2Reward：用于强化学习的语言模型奖励塑造。载于《国际学习表征会议论文集》，引用：§B.4， §4。
[207] A. Xu, M. Du, T. Yu, M. Puvvadi, T. Yu, Y. Guo, X. Chen, and J. G. Gottschlich (2025) Agentic ai for enterprise: emerging applications and real-world challenges. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pp. 6300–6301. Cited by: §1. [207] A. Xu, M. Du, T. Yu, M. Puvvadi, T. Yu, Y. Guo, X. Chen 和 J. G. Gottschlich (2025) 企业智能 AI：新兴应用与现实挑战。载于《第 31 届 ACM SIGKDD 知识发现与数据挖掘会议论文集 V. 2》，第 6300–6301 页。引用：§1。
[208] C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou, and X. Chen (2024) Large language models as optimizers. In International Conference on Learning Representations, Cited by: §B.1, §4. [208] C. Yang, X. Wang, Y. Lu, H. Liu, Q. V. Le, D. Zhou 和 X. Chen (2024) 作为优化器的大语言模型。载于《国际学习表征会议论文集》，引用：§B.1， §4。
[209] J. Yang, G. Yao, Y. Zhang, R. R. Kompella, G. Liu, and S. Chang (2026) FederatedSkill: Federated learning for agentic skill evolution. arXiv preprint arXiv:2606.03143. Cited by: §B.5, §B.5, §B.6, Table 4, §C.1, §3.2, §3.4, §4, §4, Table 1, §5.3, Table 2. [209] J. Yang, G. Yao, Y. Zhang, R. R. Kompella, G. Liu 和 S. Chang (2026) FederatedSkill：用于智能体技能演化的联邦学习。 arXiv 预印本 arXiv:2606.03143。引用：§B.5， §B.5， §B.6，表 4， §C.1， §3.2， §3.4， §4， §4，表 1， §5.3，表 2。
[210] Y. Yang, H. Chai, S. Shao, Y. Song, S. Qi, R. Rui, and W. Zhang (2026) Agentnet: decentralized evolutionary coordination for llm-based multi-agent systems. Advances in Neural Information Processing Systems 38, pp. 107309–107336. Cited by: §2. [210] Y. Yang, H. Chai, S. Shao, Y. Song, S. Qi, R. Rui 和 W. Zhang (2026) Agentnet：基于 LLM 的多智能体系统的去中心化演化协调。《神经信息处理系统进展》38，第 107309–107336 页。引用：§2。
[211] Y. Yang, G. Long, T. Shen, J. Jiang, and M. Blumenstein (2024) Dual-personalizing adapter for federated foundation models. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, pp. 39409–39433. External Links: Document, Link Cited by: §B.1, §4. [211] Y. Yang, G. Long, T. Shen, J. Jiang 和 M. Blumenstein (2024) 联邦基础模型的双重个性化适配器。载于《神经信息处理系统进展》，A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak 和 C. Zhang (编)，第 37 卷，第 39409–39433 页。外部链接：文档，链接引用：§B.1， §4。
[212] D. Yao, T. Rabbani, M. Zaheer, and T. Li (2026) Federation over text: Insight sharing for multi-agent reasoning. arXiv preprint arXiv:2604.16778. Cited by: §B.2, §B.2, §B.6, Table 4, §C.1, §3.4, §4, §4, Table 1, Table 2. [212] D. Yao, T. Rabbani, M. Zaheer 和 T. Li (2026) 文本之上的联邦：多智能体推理的洞察共享。 arXiv 预印本 arXiv:2604.16778。引用：§B.2， §B.2， §B.6，表 4， §C.1， §3.4， §4， §4，表 1，表 2。
[213] S. Yao, N. Shinn, P. Razavi, and K. Narasimhan (2025) $\tau$ -Bench: a benchmark for tool-agent-user interaction in real-world domains. In International Conference on Learning Representations, Cited by: §6.3, §7. [213] S. Yao, N. Shinn, P. Razavi 和 K. Narasimhan (2025) τ-Bench：现实领域中工具-智能体-用户交互的基准。载于《国际学习表征会议论文集》，引用：§6.3， §7。
[214] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023) ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, Cited by: §6.3. [214] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan 和 Y. Cao (2023) ReAct：协同语言模型中的推理与行动。载于《国际学习表征会议论文集》，引用：§6.3。
[215] D. Ye, T. Zhu, C. Zhu, W. Zhou, and P. S. Yu (2022) Model-based self-advising for multi-agent learning. IEEE transactions on neural networks and learning systems 34 (10), pp. 7934–7945. Cited by: §2. [215] D. Ye, T. Zhu, C. Zhu, W. Zhou 和 P. S. Yu (2022) 多智能体学习的基于模型的自我建议。《IEEE 神经网络与学习系统汇刊》34 (10)，第 7934–7945 页。引用：§2。
[216] R. Ye, W. Wang, J. Chai, D. Li, Z. Li, Y. Xu, Y. Du, Y. Wang, and S. Chen (2024) OpenFedLLM: Training large language models on decentralized private data via federated learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6137–6147. Cited by: §B.1, §4, Table 1. [216] R. Ye, W. Wang, J. Chai, D. Li, Z. Li, Y. Xu, Y. Du, Y. Wang 和 S. Chen (2024) OpenFedLLM：通过联邦学习在去中心化私有数据上训练大语言模型。载于《第 30 届 ACM SIGKDD 知识发现与数据挖掘会议论文集》，第 6137–6147 页。引用：§B.1， §4，表 1。
[217] D. Yin, Y. Chen, R. Kannan, and P. Bartlett (2018) Byzantine-robust distributed learning: towards optimal statistical rates. In International conference on machine learning, pp. 5650–5659. Cited by: §6.3. [217] D. Yin, Y. Chen, R. Kannan 和 P. Bartlett (2018) 拜占庭稳健分布式学习：迈向最优统计率。载于《国际机器学习会议论文集》，第 5650–5659 页。引用：§6.3。
[218] J. Yoon, W. Jeong, G. Lee, E. Yang, and S. J. Hwang (2021) Federated continual learning with weighted inter-client transfer. In International conference on machine learning, pp. 12073–12086. Cited by: §6.3. [218] J. Yoon, W. Jeong, G. Lee, E. Yang 和 S. J. Hwang (2021) 带有加权客户端间迁移的联邦持续学习。载于《国际机器学习会议论文集》，第 12073–12086 页。引用：§6.3。
[219] L. Yuan, Z. Zhang, L. Li, C. Guan, and Y. Yu (2023) A survey of progress on cooperative multi-agent reinforcement learning in open environment. arXiv preprint arXiv:2312.01058. Cited by: §2. [219] L. Yuan, Z. Zhang, L. Li, C. Guan, and Y. Yu (2023) A survey of progress on cooperative multi-agent reinforcement learning in open environment. arXiv preprint arXiv:2312.01058. 引用：§2.
[220] S. Yuan, K. Song, J. Chen, X. Tan, Y. Shen, K. Ren, D. Li, and D. Yang (2025) EASYTOOL: Enhancing LLM-based agents with concise tool instruction. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, New Mexico, pp. 951–972. Cited by: §B.3, §4. [220] S. Yuan, K. Song, J. Chen, X. Tan, Y. Shen, K. Ren, D. Li, and D. Yang (2025) EASYTOOL: Enhancing LLM-based agents with concise tool instruction. 载于：Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, New Mexico, pp. 951–972. 引用：§B.3, §4.
[221] Y. Yue, G. Zhang, B. Liu, G. Wan, K. Wang, D. Cheng, and Y. Qi (2025) Masrouter: learning to route llms for multi-agent systems. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15549–15572. Cited by: §2. [221] Y. Yue, G. Zhang, B. Liu, G. Wan, K. Wang, D. Cheng, and Y. Qi (2025) Masrouter: learning to route llms for multi-agent systems. 载于：Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15549–15572. 引用：§2.
[222] M. Yuksekgonul, F. Bianchi, J. Boen, S. Liu, P. Lu, Z. Huang, C. Guestrin, and J. Zou (2025) Optimizing generative AI by backpropagating language model feedback. Nature 639 (8055), pp. 609–616. Cited by: §B.1, §3.4, §4. [222] M. Yuksekgonul, F. Bianchi, J. Boen, S. Liu, P. Lu, Z. Huang, C. Guestrin, and J. Zou (2025) Optimizing generative AI by backpropagating language model feedback. Nature 639 (8055), pp. 609–616. 引用：§B.1, §3.4, §4.
[223] S. Zeng, J. Zhang, P. He, Y. Xing, Y. Liu, H. Xu, J. Ren, S. Wang, D. Yin, Y. Chang, and J. Tang (2024) The good and the bad: Exploring privacy issues in retrieval-augmented generation (RAG). In Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, pp. 4505–4524. Cited by: §B.2, §4. [223] S. Zeng, J. Zhang, P. He, Y. Xing, Y. Liu, H. Xu, J. Ren, S. Wang, D. Yin, Y. Chang, and J. Tang (2024) The good and the bad: Exploring privacy issues in retrieval-augmented generation (RAG). 载于：Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, pp. 4505–4524. 引用：§B.2, §4.
[224] Y. Zhan, J. Zhang, Z. Hong, L. Wu, P. Li, and S. Guo (2022) A survey of incentive mechanism design for federated learning. IEEE Transactions on Emerging Topics in Computing 10 (2), pp. 1035–1044. Cited by: §5.3. [224] Y. Zhan, J. Zhang, Z. Hong, L. Wu, P. Li, and S. Guo (2022) A survey of incentive mechanism design for federated learning. IEEE Transactions on Emerging Topics in Computing 10 (2), pp. 1035–1044. 引用：§5.3.
[225] B. Zhang, K. Lazuka, and M. Murag (2025) Equipping agents for the real world with Agent Skills. Note: Anthropic Engineering BlogPublished 16 October 2025. Accessed 29 September 2026 Cited by: §B.5. [225] B. Zhang, K. Lazuka, and M. Murag (2025) Equipping agents for the real world with Agent Skills. 注：Anthropic Engineering Blog，发布于2025年10月16日。访问于2026年9月29日。引用：§B.5.
[226] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao (2021) A survey on federated learning. Knowledge-based systems 216, pp. 106775. Cited by: §1, §2. [226] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao (2021) A survey on federated learning. Knowledge-based systems 216, pp. 106775. 引用：§1, §2.
[227] D. Zhang, Q. Yuan, L. Meng, R. Xia, W. Liu, and C. Qin (2025) Reinforcement learning for single-agent to multi-agent systems: from basic theory to industrial application progress, a survey. Artificial Intelligence Review 59 (2), pp. 46. Cited by: §2. [227] D. Zhang, Q. Yuan, L. Meng, R. Xia, W. Liu, and C. Qin (2025) Reinforcement learning for single-agent to multi-agent systems: from basic theory to industrial application progress, a survey. Artificial Intelligence Review 59 (2), pp. 46. 引用：§2.
[228] G. Zhang, M. Fu, K. Wang, G. Wan, M. Yu, and S. Yan (2025) G-Memory: Tracing hierarchical memory for multi-agent systems. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: §B.2, §3.4, §4. [228] G. Zhang, M. Fu, K. Wang, G. Wan, M. Yu, and S. Yan (2025) G-Memory: Tracing hierarchical memory for multi-agent systems. 载于：Advances in Neural Information Processing Systems, Vol. 38. 引用：§B.2, §3.4, §4.
[229] J. Zhang, S. Vahidian, M. Kuo, C. Li, R. Zhang, T. Yu, G. Wang, and Y. Chen (2024) Towards building the FederatedGPT: Federated instruction tuning. In 2024 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6915–6919. Cited by: §B.1, §4, Table 1. [229] J. Zhang, S. Vahidian, M. Kuo, C. Li, R. Zhang, T. Yu, G. Wang, and Y. Chen (2024) Towards building the FederatedGPT: Federated instruction tuning. 载于：2024 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6915–6919. 引用：§B.1, §4, Table 1.
[230] K. Zhang, Y. Wang, H. Wang, L. Huang, C. Yang, X. Chen, and L. Sun (2022) Efficient federated learning on knowledge graphs via privacy-preserving relation embedding aggregation. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, pp. 613–621. Cited by: §B.5, §B.5, Table 4, §4, Table 1. [230] K. Zhang, Y. Wang, H. Wang, L. Huang, C. Yang, X. Chen, and L. Sun (2022) Efficient federated learning on knowledge graphs via privacy-preserving relation embedding aggregation. 载于：Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, pp. 613–621. 引用：§B.5, §B.5, Table 4, §4, Table 1.
[231] X. Zhang, H. Gu, L. Fan, K. Chen, and Q. Yang (2022) No free lunch theorem for security and utility in federated learning. ACM Transactions on Intelligent Systems and Technology 14 (1), pp. 1–35. Cited by: §3.3. [231] X. Zhang, H. Gu, L. Fan, K. Chen, and Q. Yang (2022) No free lunch theorem for security and utility in federated learning. ACM Transactions on Intelligent Systems and Technology 14 (1), pp. 1–35. 引用：§3.3.
[232] X. Zhang, Y. Kang, K. Chen, L. Fan, and Q. Yang (2023) Trading off privacy, utility, and efficiency in federated learning. ACM Transactions on Intelligent Systems and Technology 14 (6), pp. 1–32. Cited by: §3.3. [232] X. Zhang, Y. Kang, K. Chen, L. Fan, and Q. Yang (2023) Trading off privacy, utility, and efficiency in federated learning. ACM Transactions on Intelligent Systems and Technology 14 (6), pp. 1–32. 引用：§3.3.
[233] X. Zhang, P. Zhu, C. Zhang, Z. Yan, J. Cheng, M. Lao, S. Cai, and Y. Guo (2025) Generalization-preserved learning: closing the backdoor to catastrophic forgetting in continual deepfake detection. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3798–3808. Cited by: §6.3. [233] X. Zhang, P. Zhu, C. Zhang, Z. Yan, J. Cheng, M. Lao, S. Cai, and Y. Guo (2025) Generalization-preserved learning: closing the backdoor to catastrophic forgetting in continual deepfake detection. 载于：2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3798–3808. 引用：§6.3.
[234] Z. Zhang, Q. Dai, X. Bo, C. Ma, R. Li, X. Chen, J. Zhu, Z. Dong, and J. Wen (2025) A survey on the memory mechanism of large language model-based agents. ACM Transactions on Information Systems 43 (6), pp. 155:1–155:47. Cited by: §3.1. [234] Z. Zhang, Q. Dai, X. Bo, C. Ma, R. Li, X. Chen, J. Zhu, Z. Dong, and J. Wen (2025) A survey on the memory mechanism of large language model-based agents. ACM Transactions on Information Systems 43 (6), pp. 155:1–155:47. 引用：§3.1.
[235] Z. Zhang, Y. Yang, Y. Dai, Q. Wang, Y. Yu, L. Qu, and Z. Xu (2023) FedPETuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, pp. 9963–9977. Cited by: §B.1. [235] Z. Zhang, Y. Yang, Y. Dai, Q. Wang, Y. Yu, L. Qu, and Z. Xu (2023) FedPETuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models. 载于：Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, pp. 9963–9977. 引用：§B.1.
[236] Z. Zhang and C. Zhang (2004) Agent-based hybrid intelligent systems: an agent-based framework for complex problem solving. Springer. Cited by: §1. [236] Z. Zhang and C. Zhang (2004) Agent-based hybrid intelligent systems: an agent-based framework for complex problem solving. Springer. 引用：§1.
[237] A. Zhao, D. Huang, Q. Xu, M. Lin, Y. Liu, and G. Huang (2024) ExpeL: LLM agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, pp. 19632–19642. Cited by: §B.2, §C.1, §3.1, §4. [237] A. Zhao, D. Huang, Q. Xu, M. Lin, Y. Liu, and G. Huang (2024) ExpeL: LLM agents are experiential learners. 载于：Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, pp. 19632–19642. 引用：§B.2, §C.1, §3.1, §4.
[238] D. Zhao (2024) FRAG: Toward federated vector database management for collaborative and secure retrieval-augmented generation. arXiv preprint arXiv:2410.13272. Cited by: §B.2, §4. [238] D. Zhao (2024) FRAG: Toward federated vector database management for collaborative and secure retrieval-augmented generation. arXiv preprint arXiv:2410.13272. 引用：§B.2, §4.
[239] H. Zhao, W. Du, F. Li, P. Li, and G. Liu (2023) FedPrompt: Communication-efficient and privacy-preserving prompt tuning in federated learning. In ICASSP 2023 – 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1–5. Cited by: §2. [239] H. Zhao, W. Du, F. Li, P. Li, and G. Liu (2023) FedPrompt: Communication-efficient and privacy-preserving prompt tuning in federated learning. 载于：ICASSP 2023 – 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1–5. 引用：§2.
[240] J. Zhao, H. Xie, Y. Lei, X. Song, Z. Shi, L. Li, S. Liu, and H. Zhang (2026) Weaving in the clouds: Achieving synergistic collaboration among LLM agents via federated learning. In Proceedings of the 43rd International Conference on Machine Learning, Cited by: §B.1, Table 4, §4, Table 1. [240] J. Zhao, H. Xie, Y. Lei, X. Song, Z. Shi, L. Li, S. Liu, and H. Zhang (2026) Weaving in the clouds: Achieving synergistic collaboration among LLM agents via federated learning. 载于：Proceedings of the 43rd International Conference on Machine Learning, 引用：§B.1, Table 4, §4, Table 1.
[241] W. X. Zhao, K. Zhou, J. Li, T. Tang, Z. Dong, Y. Hou, B. Zhang, Y. Min, J. Zhang, P. Liu, et al. (2026) A survey of large language models. Frontiers of Computer Science 20 (12), pp. 2012627. Cited by: §2. [241] W. X. Zhao, K. Zhou, J. Li, T. Tang, Z. Dong, Y. Hou, B. Zhang, Y. Min, J. Zhang, P. Liu, et al. (2026) A survey of large language models. Frontiers of Computer Science 20 (12), pp. 2012627. 引用：§2.
[242] B. Zheng, M. Y. Fatemi, X. Jin, Z. Z. Wang, A. Gandhi, Y. Song, Y. Gu, J. Srinivasa, G. Liu, G. Neubig, and Y. Su (2025) SkillWeaver: Web agents can self-improve by discovering and honing skills. arXiv preprint arXiv:2504.07079. Cited by: §B.5, §C.1. [242] B. Zheng, M. Y. Fatemi, X. Jin, Z. Z. Wang, A. Gandhi, Y. Song, Y. Gu, J. Srinivasa, G. Liu, G. Neubig, and Y. Su (2025) SkillWeaver: Web agents can self-improve by discovering and honing skills. arXiv preprint arXiv:2504.07079. 引用：§B.5, §C.1.
[243] L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica (2023) Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems, Vol. 36, pp. 46595–46623. Note: Datasets and Benchmarks Track Cited by: §B.4, §4. [243] L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica (2023) Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. 载于：Advances in Neural Information Processing Systems, Vol. 36, pp. 46595–46623. 注：Datasets and Benchmarks Track 引用：§B.4, §4.
[244] S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig (2024) WebArena: a realistic web environment for building autonomous agents. In International Conference on Learning Representations, Cited by: §6.3, §7. [244] S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig (2024) WebArena: a realistic web environment for building autonomous agents. 载于：International Conference on Learning Representations, 引用：§6.3, §7.
[245] C. Zhu, M. Dastani, and S. Wang (2024) A survey of multi-agent deep reinforcement learning with communication. Autonomous Agents and Multi-Agent Systems 38 (1), pp. 4. Cited by: §2. [245] C. Zhu, M. Dastani, and S. Wang (2024) A survey of multi-agent deep reinforcement learning with communication. Autonomous Agents and Multi-Agent Systems 38 (1), pp. 4. 引用：§2.
[246] L. Zhu, Z. Liu, and S. Han (2019) Deep leakage from gradients. In Advances in Neural Information Processing Systems, Vol. 32, pp. 14747–14756. Cited by: §B.1, §3.3, §4. [246] L. Zhu, Z. Liu, and S. Han (2019) Deep leakage from gradients. 载于：Advances in Neural Information Processing Systems, Vol. 32, pp. 14747–14756. 引用：§B.1, §3.3, §4.
[247] X. Zhu, G. Li, and W. Hu (2023) Heterogeneous federated knowledge graph embedding learning and unlearning. In Proceedings of the ACM Web Conference 2023, pp. 2444–2454. Cited by: §B.5, §4. [247] X. Zhu, G. Li, and W. Hu (2023) Heterogeneous federated knowledge graph embedding learning and unlearning. 载于：Proceedings of the ACM Web Conference 2023, pp. 2444–2454. 引用：§B.5, §4.
[248] M. Zhuge, C. Zhao, D. Ashley, W. Wang, D. Khizbullin, Y. Xiong, Z. Liu, E. Chang, R. Krishnamoorthi, Y. Tian, et al. (2024) Agent-as-a-judge: evaluate agents with agents. arXiv preprint arXiv:2410.10934. Cited by: §1. [248] M. Zhuge, C. Zhao, D. Ashley, W. Wang, D. Khizbullin, Y. Xiong, Z. Liu, E. Chang, R. Krishnamoorthi, Y. Tian, et al. (2024) Agent-as-a-judge: evaluate agents with agents. arXiv preprint arXiv:2410.10934. 引用：§1.
[249] Y. Zuo, Z. Xiao, L. Sheng, F. Huang, J. Tu, Y. Liu, T. Tang, X. Hu, Y. Su, Q. Lan, et al. (2026) Qwen-agentworld: language world models for general agents. arXiv preprint arXiv:2606.24597. Cited by: §1. [249] Y. Zuo, Z. Xiao, L. Sheng, F. Huang, J. Tu, Y. Liu, T. Tang, X. Hu, Y. Su, Q. Lan, et al. (2026) Qwen-agentworld: language world models for general agents. arXiv preprint arXiv:2606.24597. 引用：§1.

Appendix A Notation附录 A 符号说明

Table 3 summarizes the notation of Sections 3–5.表 3 总结了第 3–5 节中使用的符号。

Symbol	Meaning
$K$ , $k$ , $\mathcal{X}$	number and index of clients; task space shared by all clients
$\mathcal{Q}_{k}$ , $u_{k}(\tau)$	task distribution of client $k$ ; utility of a trajectory
$\mathcal{D}_{k}$ , $n_{k}$	private dataset of client $k$ (tasks, logged trajectories, raw records); its number of tasks
$A_{k}^{0}$ , $A_{k}^{t}$	initial agent of client $k$ ; its agent after round $t$
$A_{k}=(\pi_{k},\mathcal{M}_{k},\mathcal{T}_{k},\mathcal{R}_{k},\mathcal{O}_{k})$	policy, memory, tools, reward, structured knowledge (ontologies, rules, skills)
$\theta_{k}$	parameters of the policy $\pi_{k}$ , continuous or discrete
$\tau$ , $H$	trajectory and its number of steps
$U_{k}(A;\mathcal{Q}_{k})$	expected utility of agent $A$ on the tasks of client $k$
$\Phi$	protocol: participants, rules $\Update$ , $\Agg$ , $\Adapt$ , protection, schedule
$\mathcal{S}^{t}$ , $T$	clients that participate in round $t$ ; number of rounds
$\Delta_{k}=(\Delta_{k}^{\pi},\Delta_{k}^{\mathcal{M}},\Delta_{k}^{\mathcal{T}},\Delta_{k}^{\mathcal{R}},\Delta_{k}^{\mathcal{O}})$	typed update of client $k$ , each block in $\mathbb{R}^{d_{c}}\cup\Sigma^{*}\cup\mathcal{G}$
$G=\{G_{k}\}_{k=1}^{K}$	aggregated state with the view $G_{k}$ of client $k$
$A_{k}^{+}$ , $A_{k}^{\mathrm{loc}}$ , $A_{k}^{+\setminus j}$	agent of client $k$ after the federation; without the federation; without the updates of client $j$
$F=(F_{1},F_{2},F_{3})$	objectives: performance, privacy, communication
$M_{k}$	mechanism that maps $\mathcal{D}_{k}$ to the transcript of uploads of client $k$
$(\epsilon,\delta)$ ; $\mathfrak{A}$ , $\mathrm{sim}$ , $\gamma$	differential privacy parameters; threat model, similarity, and threshold of an attack
$\mathcal{P}_{k}(M_{k})$ , $\mathcal{C}_{\mathrm{comm}}(\{\Delta_{k}\},\Phi)$	privacy leakage of client $k$ , also written $\mathcal{P}_{k}(\Phi)$ ; communication cost of the protocol
$\simeq$ , $\bot$ , $\mathrm{hit}$	adjacency of datasets; no upload; success of an attack
$\lambda_{p}$ , $\lambda_{c}$ ; $\bar{\epsilon}_{k}$ , $B$	weights of the privacy and communication objectives; their budgets
$\mathcal{Z}^{\star}$ , $\mathcal{Z}_{\mathrm{w}}$ , $\mathcal{Z}_{\mathrm{c}}$	Pareto-optimal protocols; maximizers of the weighted form; of the constrained form
$\mathrm{CG}_{k}$ , $I_{j\to k}$	collaboration gain of client $k$ ; contribution of client $j$ to client $k$
$\alpha$ , $\alpha_{\ell}$ , $\rho$ , $z$	map from local evidence to an artifact; abstraction at level $\ell$ ; protection; artifact
$e\models z$ , $n_{\min}$	evidence item $e$ exhibits pattern $z$ ; support threshold
$\boldsymbol{\ell}=(\ell_{1},\dots,\ell_{K})$	levels of abstraction of all clients, part of $\Phi$
$g_{\phi}$ , $h_{\psi}$	shared evaluator and local head of the reward model
$N_{k}$ , $N_{k}^{+}$	numbers of calls and of successful calls of a tool
$\mathrm{div}$ , $\nu$ , $\mu_{k}$	divergence of task distributions; novelty threshold; alignment of concepts
$A\oplus z$ , $\hat{U}_{k}$	agent $A$ after integrating item $z$ ; utility estimated on validation tasks
$m$ , $n_{k}^{\mathrm{val}}$ , $e_{k}$ , $\eta$	number of items; number of validation tasks; error bound; failure probability

Table 3: Notation.表 3：符号说明。

Appendix B Methods that Federate a Component of an Agent附录 B 联邦化智能体组件的方法

This appendix extends Section 4. Table 4 lists the agent-level methods and selected precursors, and the following subsections describe each category together with the work on single agents on which it builds.本附录是对第 4 节的扩展。表 4 列出了智能体层面的方法及其选定的前驱研究，以下小节描述了每一类方法，以及它们所基于的单智能体研究工作。

B.1 Federated Policy OptimizationB.1 联邦策略优化

The policy block $\Delta_{k}^{\pi}$ carries parameters of the policy. Its form depends on how much access the clients have to the backbone.策略块 Δkπ 携带策略参数。其形式取决于客户端对骨干模型（backbone）的访问程度。

Weights.权重。

When the backbone is open, clients fine-tune it on local data with parameter-efficient methods and the coordinator aggregates the updates [229, 216, 235, 203]. Later work addresses clients with different adapter ranks [193, 38, 11] and the instability of adapters under differential privacy noise [171]. Personalized variants keep part of the adapter local [211, 116, 59]. At the other extreme of communication cost, zeroth-order optimization lets clients and server exchange only random seeds and scalar gradients [151].当骨干模型开放时，客户端利用参数高效方法在本地数据上对其进行微调，并由协调器聚合这些更新 [229, 216, 235, 203]。后续研究解决了客户端适配器秩（adapter rank）不同的情况 [193, 38, 11]，以及差分隐私噪声下适配器的不稳定性问题 [171]。个性化变体则将部分适配器保留在本地 [211, 116, 59]。在通信成本的另一个极端，零阶优化允许客户端与服务器仅交换随机种子和标量梯度 [151]。

The same recipe has been applied to agents. GUI agents based on vision-language models are trained on decentralized user data [186, 187, 185]. Fed-SE fine-tunes adapters on the successful trajectories of each client [35]. FedAgent trains the policy by reinforcement learning on local trajectories and averages the parameters [29]. It finds the result robust to heterogeneity of tasks but not to heterogeneity of environment dynamics. This agrees with the theory of federated reinforcement learning [144, 50, 89], in which the suboptimality of the learned policy grows with the heterogeneity of the environments [84]. FedWave federates the role-specific adapters of a workflow whose stages belong to different organizations [240]. Frameworks that separate the execution of an agent from its training make such methods applicable to existing agents [117].同样的方案也已应用于智能体。基于视觉语言模型的 GUI 智能体在去中心化的用户数据上进行训练 [186, 187, 185]。 Fed-SE 在每个客户端的成功轨迹上微调适配器 [35]。 FedAgent 通过本地轨迹上的强化学习训练策略并平均参数 [29]。研究发现，该结果对任务异构性具有鲁棒性，但对环境动态的异构性不具备鲁棒性。这与联邦强化学习的理论一致 [144, 50, 89]，即学习到的策略的次优性会随着环境异构性的增加而增加 [84]。 FedWave 联邦化了工作流中特定角色的适配器，该工作流的各个阶段属于不同的组织 [240]。将智能体执行与训练分离的框架使得此类方法适用于现有智能体 [117]。

Prompts.提示词。

When the backbone is a frozen model behind an API, the policy is optimized through its prompts. Clients can share continuous prompts [61], optimize prompts without gradients by querying the model [170, 113], or share textual prompts that the coordinator merges by summarization [30]. The last option connects FAO to the optimization of prompts and agentic workflows [208, 88, 222, 3], where the variables are text and the optimizer is often itself a language model.当骨干模型是 API 后端的冻结模型时，策略通过其提示词（prompts）进行优化。客户端可以共享连续提示词 [61]，通过查询模型在无梯度的情况下优化提示词 [170, 113]，或者共享由协调器通过摘要合并的文本提示词 [30]。最后一种选择将 FAO（联邦智能体优化）与提示词和智能体工作流的优化联系起来 [208, 88, 222, 3]，其中变量是文本，而优化器本身通常也是一个语言模型。

Trade-offs.权衡。

Weight sharing is the most mature form and the most expensive in communication: Fed-SE reports 76.3 MB per round for adapters of rank 8 [35]. Its leakage is well studied. Gradients can be inverted to recover training text, for recent attacks exactly and for large batches [246, 64, 139], and restricting the update to adapters does not remove the risk [163]. Secure aggregation and differentially private training are the standard defenses [18, 1]. Prompt sharing has a small payload and works with closed backbones, but a textual prompt is readable by anyone who receives it and may quote the examples from which it was derived.权重共享是最成熟的形式，也是通信成本最高的形式：Fed-SE 报告称，对于秩为 8 的适配器，每轮通信量为 76.3 MB [35]。其泄露风险已得到充分研究。梯度可以被反转以恢复训练文本，对于最近的攻击，即使是大批量数据也能实现精确恢复 [246, 64, 139]，且将更新限制在适配器上并不能消除这种风险 [163]。安全聚合和差分隐私训练是标准防御手段 [18, 1]。提示词共享的有效载荷较小，且适用于闭源骨干模型，但文本提示词对于任何接收者都是可读的，并且可能会引用从中派生出来的示例。

Method	Block	Shared update	Aggregation
Agent-level methods
FedMABench [186], FedGUI [185]	$\Delta^{\pi}$	adapters of GUI agents based on vision-language models	standard federated algorithms
MobileA3gent [187]	$\Delta^{\pi}$	models trained on automatically annotated phone usage	averaging weighted by episode and step statistics
Fed-SE^∗ [35]	$\Delta^{\pi}$	adapters tuned on successful trajectories	unweighted averaging of adapters
FedAgent^∗ [29]	$\Delta^{\pi}$	parameters trained by reinforcement learning	federated averaging
FedWave [240]	$\Delta^{\pi}$	role-specific adapters of a multi-stage workflow	federated adapters fused by a shared mixture-of-experts router
FoT^∗ [212]	$\Delta^{\mathcal{M}}$	summaries of reasoning traces	clustering and distillation into an insight library
FedAgentKE^∗ [109]	$\Delta^{\mathcal{M}}$	knowledge units with workflows, corrections, and utility scores	clustering, selection, deduplication
FICAL^∗ [200]	$\Delta^{\mathcal{T}}$	textual knowledge on tool use	concatenation
Typed artifacts^∗ [25]	$\Delta^{\mathcal{T}}$	schema-validated tool-routing knowledge	field-wise merging
FedWorld^∗ [75]	$\Delta^{\mathcal{O}}$	abstract transition rules	alignment of rules; scope inferred from supporting and contradicting evidence
FederatedSkill^∗ [209]	$\Delta^{\mathcal{O}}$	patches to the local skill library	merging and rewriting by a server-side agent, separately for each client
Selected precursors
FedTextGrad [30]	$\Delta^{\pi}$	textual prompts of a language model	concatenation or summarization by a language model
FedRLHF [51]	$\Delta^{\pi}$	policies trained on local rewards from human feedback, which stay local	federated aggregation of the policies
FedE4RAG^∗ [121]	$\Delta^{\mathcal{M}}$	encrypted parameters of retrievers	averaging with distillation between server and clients
FedBiscuit [199]	$\Delta^{\mathcal{R}}$	binary selectors that encode client preferences	aggregation within groups of similar clients
FedGD^∗ [90]	$\Delta^{\mathcal{R}}$	parameters of a reward model	averaging with group-debiased client sampling
FedR [230]	$\Delta^{\mathcal{O}}$	relation embeddings of a knowledge graph	secure aggregation over aligned relations

Table 4: Agent-level FAO methods and selected precursors, as instances of the typed update of Eq. (5). ^∗Preprint.表 4：智能体层面的 FAO 方法及选定的前驱研究，作为式 (5) 类型化更新的实例。*预印本。

B.2 Federated Memory OptimizationB.2 联邦记忆优化

The memory block $\Delta_{k}^{\mathcal{M}}$ carries experience in abstracted form. Raw trajectories are the most informative content of memory and the most sensitive, so they stay local. What can be shared are the products that agents already derive from trajectories for their own use: reflections on failures [167], insights extracted from successes and failures [237], workflows induced from repeated tasks [194], and reasoning strategies [134].记忆块 Δkℳ 携带抽象形式的经验。原始轨迹是记忆中信息量最大且最敏感的内容，因此它们保留在本地。可以共享的是智能体已经从轨迹中为自身用途导出的产物：对失败的反思 [167]、从成功和失败中提取的见解 [237]、从重复任务中归纳出的工作流 [194] 以及推理策略 [134]。

Several systems let agents share such items through a common store: a pool of prompt–answer pairs [53], a hierarchical memory for the agents of one system [228], and a knowledge base of workflows and fixes built from the trajectories of different agent frameworks [174]. Access control over memory that is shared among users has also been studied [157]. These systems assume that the store is trusted.多个系统允许智能体通过公共存储共享此类条目：提示词-回答对池 [53]、单个系统的智能体分层记忆 [228]，以及由不同智能体框架的轨迹构建的工作流和修复知识库 [174]。用户间共享记忆的访问控制也已得到研究 [157]。这些系统假设存储是受信任的。

Federated methods avoid centralizing raw trajectories in such a store. In FoT, each client uploads summaries of its reasoning traces, without the problem instances or task instructions, and the server distills them into a library of insights that all clients can use [212]. FedAgentKE exchanges knowledge units that contain reusable workflows and corrections together with a utility score, and each client retrieves the units relevant to its task [109]. For document collections, federated retrieval-augmented generation [24] answers queries over the collections of several providers without centralizing the documents, protected by confidential computing [2] or homomorphic encryption [238], and it trains the retrievers of the clients by federated learning [121].联邦方法避免了在此类存储中集中原始轨迹。在 FoT 中，每个客户端上传其推理轨迹的摘要，不包含问题实例或任务指令，服务器将其提炼成所有客户端都可以使用的见解库 [212]。 FedAgentKE 交换包含可重用工作流和修正的知识单元以及效用分数，每个客户端检索与其任务相关的单元 [109]。对于文档集合，联邦检索增强生成 [24] 在不集中文档的情况下回答多个提供商集合上的查询，由机密计算 [2] 或同态加密 [238] 保护，并通过联邦学习训练客户端的检索器 [121]。

Aggregation.聚合。

Memory items are text, so they cannot be averaged. The coordinator clusters the items, removes duplicates, merges items that express the same pattern, and ranks the result by utility or by the number of clients that support it. A language model performs these operations, which makes the coordinator an agent in its own right.记忆条目是文本，因此无法进行平均。协调器对条目进行聚类、删除重复项、合并表达相同模式的条目，并按效用或支持该条目的客户端数量对结果进行排序。语言模型执行这些操作，这使得协调器本身也成为一个智能体。

Trade-offs.

The payload is small: in one setting of FoT, the traces of all clients amount to 33K tokens and the distilled library to 3K tokens [212]. The leakage is of a different kind than for weights. An item written in natural language may quote private content, retrieval stores and agent memories can be extracted by crafted queries [223, 179], and embeddings can be inverted [128]. Protection therefore has to act on the text. FoT audits its uploads with a reconstruction attack [212]. Differentially private generation of demonstrations [175, 202] and of synthetic text [204] would add a formal guarantee, but to our knowledge it has not yet been applied to the memory of agents.有效载荷很小：在 FoT 的一种设置中，所有客户端的轨迹总计 33K 个 token，提炼出的库为 3K 个 token [212]。其泄露性质与权重不同。用自然语言编写的条目可能会引用私有内容，检索存储和智能体记忆可以通过精心设计的查询提取 [223, 179]，嵌入向量也可以被反转 [128]。因此，保护措施必须作用于文本。 FoT 使用重构攻击对其上传内容进行审计 [212]。差分隐私生成的演示 [175, 202] 和合成文本 [204] 将增加形式化保证，但据我们所知，它尚未应用于智能体的记忆。

B.3 Federated Tool-Use OptimizationB.3 联邦工具使用优化

The tool block $\Delta_{k}^{\mathcal{T}}$ carries knowledge about how to use tools. In the horizontal setting the clients use tools of the same kind, such as customer databases, payment interfaces, or document recognition, and standard interfaces such as the Model Context Protocol [8] make the descriptions of tools comparable across clients.工具块 Δk𝒯 携带关于如何使用工具的知识。在水平设置中，客户端使用同类工具，例如客户数据库、支付接口或文档识别，并且模型上下文协议（Model Context Protocol）[8] 等标准接口使工具描述在客户端之间具有可比性。

Research on single agents shows what this knowledge consists of [149, 153]. Models learn to call tools from examples of correct calls [165, 137, 150]. Concise and consistent descriptions improve the use of a tool [220], and a description can be refined from the experience of calling the tool [152]. Agents can also create new tools [20, 147, 189].关于单智能体的研究展示了这些知识的构成 [149, 153]。模型通过正确调用的示例学习调用工具 [165, 137, 150]。简洁且一致的描述可以改善工具的使用 [220]，并且描述可以从调用工具的经验中进行细化 [152]。智能体也可以创建新工具 [20, 147, 189]。

Two methods federate such knowledge. In FICAL, each client writes a compendium that describes its tools, the scenarios in which to use them, precautions, and the coordination of several tools; the server concatenates the compendiums, and the agents retrieve from the result [200]. Typed federated artifacts replace free text by schema-validated objects with separate fields for privacy handling, conflict resolution, and transfer across models, and they are designed for frozen models of different vendors [25]. Usage statistics, such as the success rate of a tool for a given intent, are a third kind of tool knowledge. Only their sum is needed, so they can be protected by secure aggregation.两种方法联邦化了此类知识。在 FICAL 中，每个客户端编写一份描述其工具、使用场景、注意事项以及多个工具协调的纲要；服务器连接这些纲要，智能体从结果中进行检索 [200]。类型化联邦工件（Typed federated artifacts）用模式验证对象替换自由文本，这些对象具有用于隐私处理、冲突解决和跨模型传输的独立字段，专为不同厂商的冻结模型设计 [25]。使用统计数据（例如给定意图下工具的成功率）是第三种工具知识。由于只需要它们的总和，因此可以通过安全聚合来保护它们。

Trade-offs.

Tool knowledge in text is small compared with model weights: FICAL reports 0.078 MB against 26,000 MB for baselines that exchange parameters [200]. It is not small by nature, however, and a compendium of tool-routing experience can reach 20 MB per client and round [25]. The leakage concerns the systems of the client more than its data: tool schemas disclose the structure of internal systems, and the arguments of recorded calls may contain customer records. Shared tool knowledge is also an attack surface. A tool description is an instruction to the agent that reads it, and threats from malicious tool providers are documented for the Model Context Protocol [74]. Aggregation of tool knowledge therefore needs robustness against malicious contributions in addition to privacy. Evaluation is a further open issue: the study of typed artifacts reports that a simple retrieval baseline outperforms all of its routing variants based on language models, and concludes that the benchmark cannot measure the benefit of shared routing knowledge [25].文本形式的工具知识与模型权重相比很小：FICAL 报告为 0.078 MB，而交换参数的基线为 26,000 MB [200]。然而，它本质上并不小，工具路由经验的纲要每个客户端每轮可以达到 20 MB [25]。泄露问题更多地涉及客户端的系统而非其数据：工具模式会披露内部系统的结构，记录的调用参数可能包含客户记录。共享工具知识也是一个攻击面。工具描述是读取它的智能体的指令，恶意工具提供商带来的威胁在模型上下文协议中已有记录 [74]。因此，工具知识的聚合除了隐私之外，还需要对恶意贡献具有鲁棒性。评估是另一个悬而未决的问题：类型化工件的研究报告称，简单的检索基线优于其所有基于语言模型的路由变体，并得出结论：该基准测试无法衡量共享路由知识的好处 [25]。

B.4 Federated Reward OptimizationB.4 联邦奖励优化

The reward block $\Delta_{k}^{\mathcal{R}}$ carries knowledge about how to judge a trajectory. The problem has a natural division of labor. Clients observe real outcomes and human judgments, but each client has few of them. The coordinator can run a strong evaluator, but language models used as judges show systematic biases [243], and on the trajectories of web agents no single judge is reliable across benchmarks [115].奖励块 Δkℛ 携带关于如何判断轨迹的知识。这个问题存在自然的分工。客户端观察真实结果和人类判断，但每个客户端拥有的数量很少。协调器可以运行一个强大的评估器，但用作裁判的语言模型表现出系统性偏差 [243]，并且在 Web 智能体的轨迹上，没有单一的裁判在所有基准测试中都可靠 [115]。

Federated preference and reward models.联邦偏好和奖励模型。

Existing methods address the alignment of language models with human preferences [39]. In FedRLHF, the feedback and the reward function stay with the client, and the policies are federated [51]. FedBis and FedBiscuit federate the preference model itself: each client encodes its preferences in a binary selector, and the selectors are aggregated [199]. Reward models and preference predictors are also trained by federated averaging and then personalized by local fine-tuning [168, 90], and a federation of client models can serve as a panel of referees [70]. To our knowledge, no method yet federates an evaluator of agent trajectories.现有方法解决了语言模型与人类偏好对齐的问题 [39]。在 FedRLHF 中，反馈和奖励函数保留在客户端，策略进行联邦化 [51]。 FedBis 和 FedBiscuit 联邦化了偏好模型本身：每个客户端将其偏好编码在二进制选择器中，并聚合这些选择器 [199]。奖励模型和偏好预测器也通过联邦平均进行训练，然后通过本地微调进行个性化 [168, 90]，客户端模型联盟可以充当裁判小组 [70]。据我们所知，目前还没有方法联邦化智能体轨迹的评估器。

Building blocks for agents.智能体的构建块。

Three lines of work on single agents provide the pieces. Evaluators can be trained or prompted to judge the success of web navigation and device control [136]. Since feedback arrives at the end of a trajectory, credit must be assigned to individual steps [140]; process reward models, which score each step, serve this purpose [110, 23]. Language models can write reward functions from a description of the task [94, 119, 206]. The last line suggests a form of federation with very low cost: a language model at the coordinator proposes a reward function or a rubric, each client scores the proposal against its private outcomes, and only the scores return to the coordinator.关于单智能体的三条研究路线提供了这些部分。评估器可以被训练或提示以判断 Web 导航和设备控制的成功 [136]。由于反馈在轨迹结束时到达，因此必须将信用分配给各个步骤 [140]；对每一步进行评分的过程奖励模型（process reward models）实现了这一目的 [110, 23]。语言模型可以根据任务描述编写奖励函数 [94, 119, 206]。最后一条路线提出了一种成本极低的联邦形式：协调器处的语言模型提出奖励函数或准则，每个客户端根据其私有结果对提议进行评分，只有分数返回给协调器。

Trade-offs.

The utilities $u_{k}$ differ across clients, so a single global evaluator may be wrong for every client. The methods above respond by grouping clients with similar preferences [199] or by learning an initialization that adapts quickly to each client [90]. The main leakage channel is the label: a preference or an outcome label reveals how an individual case ended.效用 uku 在客户端之间不同，因此单一的全局评估器可能对每个客户端都是错误的。上述方法通过将具有相似偏好的客户端分组 [199] 或通过学习能够快速适应每个客户端的初始化 [90] 来进行响应。主要的泄露渠道是标签：偏好或结果标签揭示了个体案例的结局。

B.5 Federated Structured Knowledge and Skill OptimizationB.5 联邦结构化知识和技能优化

The block $\Delta_{k}^{\mathcal{O}}$ carries structured knowledge: ontologies and knowledge graphs, rules on the business and the environment, and skills.块 Δk𝒪 携带结构化知识：本体和知识图谱、关于业务和环境的规则，以及技能。

Ontologies.本体。

Federated learning of knowledge graph embeddings lets clients with overlapping entities improve their embeddings without exchanging triples. In FedE, clients upload entity embeddings and the server averages them [31]. Entity embeddings, however, allow private triples to be reconstructed, and FedR therefore aggregates only relation embeddings, under secure aggregation, which lowers both leakage and communication cost [230]. FKGE needs no server: pairs of clients translate the embeddings of aligned entities and relations into each other’s space through an adversarial network with a differential privacy guarantee [138]. FedLU keeps the local embeddings apart from the global ones and transfers knowledge between them by distillation [247].知识图谱嵌入的联邦学习允许具有重叠实体的客户端在不交换三元组的情况下改进其嵌入。在 FedE 中，客户端上传实体嵌入，服务器对其进行平均 [31]。然而，实体嵌入允许私有三元组被重构，因此 FedR 在安全聚合下仅聚合关系嵌入，这降低了泄露和通信成本 [230]。 FKGE 不需要服务器：客户端对通过具有差分隐私保证的对抗网络，将对齐实体和关系的嵌入转换为彼此的空间 [138]。 FedLU 将本地嵌入与全局嵌入分开，并通过蒸馏在它们之间传递知识 [247]。

These methods assume that the entities and relations of the clients have been aligned beforehand. Ontology matching is a classical problem [48]. Language models and agents now reach the level of established matching systems on standard benchmarks and improve on them in complex cases [71, 10, 148], and the combination of ontologies and language models has been used to harmonize data before federated learning [91].这些方法假设客户端的实体和关系已预先对齐。本体匹配是一个经典问题 [48]。语言模型和智能体现在在标准基准测试上达到了既定匹配系统的水平，并在复杂情况下优于它们 [71, 10, 148]，并且本体和语言模型的组合已被用于在联邦学习之前协调数据 [91]。

Rules.规则。

FedWorld federates what agents learn about the dynamics of their environment. Each client converts its private transitions into abstract rules. The server aligns the rules of different clients, collects the evidence that supports and contradicts each rule, and labels the rule as shared, specific to a cluster of clients, private, or unresolved [75].FedWorld 联邦化了智能体对其环境动态的学习成果。每个客户端将其私有转换转换为抽象规则。服务器对齐不同客户端的规则，收集支持和反驳每条规则的证据，并将规则标记为共享、特定于客户端集群、私有或未解决 [75]。

Skills.技能。

A skill packages a procedure so that an agent can load it when needed [180, 242, 225]. In FederatedSkill, clients upload patches to their skill libraries, from which task-specific values are omitted, and an agent at the server merges and rewrites the patches into a library for each client [209].技能封装了一个过程，以便智能体可以在需要时加载它 [180, 242, 225]。在 FederatedSkill 中，客户端上传对其技能库的补丁，从中省略任务特定的值，服务器处的智能体将补丁合并并重写为每个客户端的库 [209]。

Trade-offs.

An aligned ontology helps the other categories, since insights, tool descriptions, and rubrics all refer to business concepts. It is also expensive to build and changes slowly, so its communication cost is incurred mostly once. Skills are cheap to exchange: FederatedSkill reports between 99 and 281 KB for a family of tasks over a complete run [209]. The leakage concerns existence and process: the entity embeddings of a client reveal which entities it knows and can expose the relations among them [230], and a skill reveals how a client conducts its business.对齐的本体有助于其他类别，因为见解、工具描述和准则都涉及业务概念。构建它也很昂贵且变化缓慢，因此其通信成本大多是一次性的。技能交换成本低廉：FederatedSkill 报告称，在完整运行过程中，一系列任务的交换量在 99 到 281 KB 之间 [209]。泄露问题涉及存在性和过程：客户端的实体嵌入揭示了它知道哪些实体并可能暴露它们之间的关系 [230]，而技能揭示了客户端如何开展业务。

B.6 ComparisonB.6 比较

The five categories occupy different regions of the objective space of Eq. (10).这五个类别占据了式 (10) 目标空间的不同区域。

Communication.通信。

The reported payloads differ by roughly three orders of magnitude, from tens of megabytes per round for adapters to a few hundred kilobytes per run for skills, although the studies report them at different granularities. The order of the categories is not fixed. Accumulated text can grow large, and parameter updates can be compressed to a few kilobytes [151].报告的有效载荷相差约三个数量级，从适配器每轮几十 MB 到技能每运行几百 KB 不等，尽管研究在不同粒度上报告了它们。类别的顺序不是固定的。积累的文本可能会变得很大，而参数更新可以压缩到几 KB [151]。

Privacy.隐私。

Weight sharing has the best understood attacks and defenses. The methods that exchange text or rules [200, 212, 109, 209, 75] rely on the fact that raw trajectories stay local, and none of them applies a formal privacy mechanism. Two of them support the argument by an empirical audit with reconstruction attacks [212, 209]. The privacy problem thus shifts from the inversion of numerical updates to the disclosure of content in text, for which formal tools are less developed.权重共享具有最易理解的攻击和防御手段。交换文本或规则的方法 [200, 212, 109, 209, 75] 依赖于原始轨迹保留在本地这一事实，且都没有应用形式化的隐私机制。其中两个通过重构攻击的经验审计来支持这一论点 [212, 209]。因此，隐私问题从数值更新的反转转向了文本内容的披露，而针对后者的形式化工具尚不成熟。

Access to the backbone.骨干模型访问权限。

Weight sharing requires open weights and a common architecture. The sharing of prompts, memory items, tool knowledge, rubrics, rules, and skills remains possible when every client uses a different closed model.权重共享需要开放权重和通用架构。当每个客户端使用不同的闭源模型时，共享提示词、记忆条目、工具知识、准则、规则和技能仍然是可能的。

Maturity.成熟度。

Federated fine-tuning and federated knowledge graph embedding are established. For rewards, methods exist for the alignment of language models but not yet for agent trajectories. For memory, tool use, rules, and skills, the first federated methods have appeared, most of them as preprints. Each of these methods optimizes performance for one component and treats privacy and communication as properties of its design. To our knowledge, none formulates them as objectives to be traded off against performance, across components, as Eq. (10) does.联邦微调和联邦知识图谱嵌入已经建立。对于奖励，存在语言模型对齐的方法，但对于智能体轨迹尚不存在。对于记忆、工具使用、规则和技能，第一批联邦方法已经出现，其中大多数为预印本。这些方法中的每一种都为一个组件优化了性能，并将隐私和通信视为其设计的属性。据我们所知，没有一种方法像式 (10) 那样，将它们表述为跨组件与性能进行权衡的目标。

Appendix C Knowledge Types in the Running Example附录 C 运行示例中的知识类型

This appendix extends Section 5.2. We use one running example. Several banks deploy agents for customer identity verification. Each agent talks to applicants, calls tools for document recognition and registry lookup, and decides whether an application passes, fails, or is escalated to a human reviewer. Customer records and conversation logs cannot leave the bank.本附录是对第 5.2 节的扩展。我们使用一个运行示例。几家银行部署了用于客户身份验证的智能体。每个智能体与申请人交谈，调用工具进行文档识别和注册查询，并决定申请是通过、失败还是升级到人工审核。客户记录和对话日志不能离开银行。

C.1 The Four Types in the Running ExampleC.1 运行示例中的四种类型

Experience.经验。

The sender shares what it has learned from successes and failures, stated as a pattern and not as a case. In the running example, one bank observes that applications fail at the matching step when the name on the document and the name in the registry are transliterated differently, and that normalizing the transliteration before the lookup avoids the failure. The shared artifact is this insight. Methods that distill such insights from the trajectories of a single agent are well developed [167, 237, 134]. In a federation, a first distillation runs at the client, so that only summaries are uploaded, and a second one at the coordinator, which condenses the summaries of all clients into a shared library [212, 109].发送者共享其从成功和失败中学到的东西，以模式而非案例的形式陈述。在运行示例中，一家银行观察到，当文档上的名称与注册表中的名称音译不同时，申请在匹配步骤失败，而在查询前规范化音译可以避免失败。共享的工件就是这一见解。从单个智能体的轨迹中提炼此类见解的方法已很成熟 [167, 237, 134]。在联邦中，第一次提炼在客户端运行，因此只上传摘要，第二次在协调器处运行，将所有客户端的摘要浓缩成一个共享库 [212, 109]。

Skills.

The sender shares a reusable procedure. A skill for verifying a business license may consist of four steps: recognize the document, query the registry by license number, compare the legal representative, and escalate on mismatch. The skill is parameterized, so that the receiver binds each step to its own tools. Skills can be induced from trajectories as workflows or programs [180, 194, 242], and they can be exchanged as patches to a skill library [209]. A skill carries more structure than an insight and is more directly executable, but it also discloses more of the sender’s business process.发送者共享一个可重用的过程。验证营业执照的技能可能包括四个步骤：识别文档、按许可证号查询注册表、比较法定代表人、在不匹配时升级。该技能是参数化的，因此接收者将每一步绑定到自己的工具。技能可以从轨迹中归纳为工作流或程序 [180, 194, 242]，并且可以作为技能库的补丁进行交换 [209]。技能比见解携带更多的结构，执行起来也更直接，但也更多地披露了发送者的业务流程。

Rewards.奖励。

The sender shares what it has learned about how to judge a trajectory. Clients hold the ground truth, such as the decisions of human reviewers, but each has few labels. The coordinator may hold a stronger evaluator but has no access to real outcomes. Federated training of the evaluator combines the two. It has been developed for preference models of language models [199, 90] and remains open for agent trajectories (Section B.4). A shared rule in the running example is that a verification dialogue is non-compliant if the agent tells the applicant which field failed the check. An improved evaluator benefits every later optimization step of the receiver, since it supplies the signal on which the policy, the memory, and the skills are tuned.发送者共享其关于如何判断轨迹的知识。客户端持有真实情况，例如人工审核员的决定，但每个客户端的标签很少。协调器可能拥有更强的评估器，但无法访问真实结果。评估器的联邦训练结合了两者。它已为语言模型的偏好模型开发 [199, 90]，对于智能体轨迹仍处于开放状态（第 B.4 节）。运行示例中的共享规则是，如果智能体告诉申请人哪个字段检查失败，则验证对话是不合规的。改进的评估器有利于接收者的每一次后续优化步骤，因为它提供了策略、记忆和技能调整的信号。

Ontologies.

The sender shares how its concepts relate to those of others. One bank records the beneficial owner of a company, and another records the ultimate controlling person; an alignment states whether the two are the same concept and under which conditions. Language models and agents are increasingly used to find such correspondences [148], and once concepts are aligned, their embeddings can be improved jointly without exchanging the underlying records [31, 138].发送者共享其概念如何与他人的概念相关联。一家银行记录公司的受益所有人，另一家记录最终控制人；对齐说明两者是否为同一概念以及在什么条件下。语言模型和智能体越来越多地被用于寻找此类对应关系 [148]，一旦概念对齐，就可以在不交换基础记录的情况下共同改进它们的嵌入 [31, 138]。

Rules.

The sender shares what it has learned about the effects of actions. In the running example, one bank finds that three failed one-time codes lock the account of an applicant for 30 minutes, so that an agent should escalate instead of retrying. FedWorld federates such knowledge as abstract transition rules and labels each rule as shared, specific to a cluster of clients, private, or unresolved [75].发送者共享其学到的关于动作效果的知识。在运行示例中，一家银行发现三次失败的一次性代码会锁定申请人的账户 30 分钟，因此智能体应该升级而不是重试。 FedWorld 将此类知识联邦化为抽象转换规则，并将每条规则标记为共享、特定于客户端集群、私有或未解决 [75]。

Interactions among the types.类型间的相互作用。

The five types differ in function and are not ordered by abstraction. Ontology alignment makes insights, skills, and rules easier to transfer, because these artifacts are written in terms of concepts that the receiver must ground, although natural language often aligns concepts implicitly. The reward type acts at both ends of the loop: at the sender it selects which trajectories are worth distilling, and at the receiver it can estimate whether an adopted item helps when observed outcomes are scarce. A federation without aligned concepts or a reliable evaluator finds it harder to tell useful knowledge from noise.这五种类型在功能上有所不同，并非按抽象程度排序。本体对齐使见解、技能和规则更容易转移，因为这些工件是根据接收者必须扎根的概念编写的，尽管自然语言通常隐式对齐概念。奖励类型在循环的两端起作用：在发送者处，它选择哪些轨迹值得提炼；在接收者处，当观察到的结果稀缺时，它可以估计采用的条目是否有帮助。没有对齐概念或可靠评估器的联邦更难从噪声中分辨有用的知识。

C.2 From Shared Knowledge to Local ImprovementC.2 从共享知识到本地改进

The type of shared knowledge does not determine the component in which the receiver stores it. An insight can be written to memory and retrieved when a similar task arrives, added to the system prompt, used as a training example for the policy, or used as a negative example for the evaluator. Table 5 lists the combinations. The relation between the types of this section and the categories of Section 4 is therefore many-to-many: the taxonomy describes what the federation maintains, and the types describe what travels between clients.共享知识的类型并不决定接收者将其存储在哪个组件中。见解可以写入记忆并在任务到达时检索，添加到系统提示词中，用作策略的训练示例，或用作评估器的负面示例。表 5 列出了这些组合。因此，本节的类型与第 4 节的类别之间的关系是多对多的：分类法描述了联邦维护的内容，而类型描述了在客户端之间传递的内容。

Shared type	Policy $\pi_{k}$	Memory $\mathcal{M}_{k}$	Tools $\mathcal{T}_{k}$	Reward $\mathcal{R}_{k}$	Structured knowledge $\mathcal{O}_{k}$
Experience	demonstrations for prompting or fine-tuning	insights stored for retrieval	corrected tool descriptions, known failure cases	positive and negative examples for the evaluator	new rules derived from recurring patterns
Skill	procedures placed in the prompt	procedures retrieved on demand	compositions of tools	checkpoints that a correct trajectory must pass	extension of the skill library
Rule	preconditions of actions stated in the prompt	rules retrieved for similar states	expected effects and failure modes of tool calls	penalties for actions with known bad effects	extension of the rule base
Reward	training signal for policy optimization	filter for what is written to memory	reliability scores of tools	parameters or rubrics of the evaluator	validation of new rules and skills
Ontology	shared vocabulary in prompts	retrieval keys over aligned concepts	mapping of tool arguments to shared concepts	criteria stated over shared concepts	aligned concepts and relations

Table 5: How knowledge of each type (rows) can improve each component of the receiving agent (columns). One shared artifact can improve several components.表 5：每种类型的知识（行）如何改进接收智能体的每个组件（列）。一个共享工件可以改进多个组件。

Appendix D Extensions of the Formulation附录 D 公式的扩展

Three further concerns fit into the same formulation. Inference cost (tokens, latency, and fees for tool calls) can be charged inside $u_{k}$ or added as a fourth objective. Non-stationarity can be modeled by letting the task distribution $\mathcal{Q}_{k}^{t}$ depend on the round, in which case utility is evaluated on the current distribution and the agent must adapt without losing capabilities it still needs.另外三个关注点也符合相同的公式。推理成本（token、延迟和工具调用费用）可以在 uku 中收费，或作为第四个目标添加。非平稳性可以通过让任务分布 𝒬kt 依赖于轮次来建模，在这种情况下，效用在当前分布上进行评估，智能体必须在不丢失其仍需的能力的情况下进行适应。

Protocols without a coordinator, such as the exchange of embeddings between pairs of clients in FKGE [138], compute $\Agg$ among the clients. They are covered if the communication cost counts every message $\mathrm{msg}$ in the set $\mathcal{C}_{t}(\Phi)$ of messages that clients exchange in round $t$ ,没有协调器的协议，例如 FKGE 中客户端对之间的嵌入交换 [138]，在客户端之间计算 Agg。如果通信成本计算轮次 t 中客户端交换的消息集合 𝒞t(Φ) 中的每条消息 msg，则它们也被涵盖，

\mathcal{C}_{\mathrm{comm}}(\Phi)=\sum_{t=0}^{T-1}\sum_{\mathrm{msg}\in\mathcal{C}_{t}(\Phi)}|\mathrm{msg}|,

(32)

which reduces to Eq. (8) for the star topology with a coordinator that the main text considers.这对于正文考虑的带有协调器的星型拓扑简化为式 (8)。

Appendix E Proofs附录 E 证明

Proposition 1 (Scalarized forms).命题 1（标量化形式）。

Assume that the maximizers of Eqs. (13) and (14) exist. Every maximizer of Eq. (13) with $\lambda_{p},\lambda_{c}>0$ is Pareto optimal, and every Pareto-optimal protocol $\Phi^{\star}$ maximizes Eq. (14) with the budgets $\bar{\epsilon}_{k}=\mathcal{P}_{k}(\Phi^{\star})$ and $B=\mathcal{C}_{\mathrm{comm}}(\Phi^{\star})$ .假设式 (13) 和 (14) 的最大化器存在。式 (13) 中 λp,λc>0 的每个最大化器都是帕累托最优的，并且每个帕累托最优协议 Φ⋆ 最大化了式 (14)，其预算为 ϵ¯k=𝒫k(Φ⋆) 和 B=𝒞comm(Φ⋆)。

Proof.证明。

Eq. (13) maximizes $w(\Phi)=F_{1}(\Phi)+\lambda_{p}F_{2}(\Phi)+\lambda_{c}F_{3}(\Phi)$ . If a maximizer $\Phi$ were dominated by some $\Phi^{\prime}$ , then $F_{i}(\Phi^{\prime})\geq F_{i}(\Phi)$ for all $i$ with at least one strict inequality, and since all weights are positive, $w(\Phi^{\prime})>w(\Phi)$ , a contradiction. For the second claim, $\Phi^{\star}$ is feasible for the stated budgets. If a feasible $\Phi^{\prime}$ had $F_{1}(\Phi^{\prime})>F_{1}(\Phi^{\star})$ , then $\mathcal{P}_{k}(\Phi^{\prime})\leq\mathcal{P}_{k}(\Phi^{\star})$ for all $k$ would give $F_{2}(\Phi^{\prime})\geq F_{2}(\Phi^{\star})$ , and $\mathcal{C}_{\mathrm{comm}}(\Phi^{\prime})\leq\mathcal{C}_{\mathrm{comm}}(\Phi^{\star})$ would give $F_{3}(\Phi^{\prime})\geq F_{3}(\Phi^{\star})$ , so $\Phi^{\prime}$ would dominate $\Phi^{\star}$ , a contradiction. ∎式 (13) 最大化 w(Φ)=F1(Φ)+λpF2(Φ)+λcF3(Φ)。如果最大化器 Φ 被某个 Φ′ 支配，则对于所有 i，Fi(Φ′)≥Fi(Φ)，且至少有一个严格不等式，由于所有权重均为正，w(Φ′)>w(Φ)，这产生矛盾。对于第二个声明，Φ⋆ 对于所述预算是可行的。如果一个可行的 Φ′ 具有 F1(Φ′)>F1(Φ⋆)，则对于所有 k，𝒫k(Φ′)≤𝒫k(Φ⋆) 将给出 F2(Φ′)≥F2(Φ⋆)，且 𝒞comm(Φ′)≤𝒞comm(Φ⋆) 将给出 F3(Φ′)≥F3(Φ⋆)，因此 Φ′ 将支配 Φ⋆，这产生矛盾。 ∎

The constrained form bounds the leakage of every client, which is stronger than a bound on the average $F_{2}$ , and it still reaches every Pareto-optimal protocol. The converse inclusion fails in general, since a maximizer of Eq. (14) may be only weakly Pareto optimal [124].约束形式限制了每个客户端的泄露，这比平均 F2 的界限更强，并且它仍然涵盖了每个帕累托最优协议。反向包含通常不成立，因为式 (14) 的最大化器可能仅是弱帕累托最优的 [124]。

Proposition 2 (Gated adaptation).命题 2（门控适应）。

Let the items $z_{1},\dots,z_{m}$ be fixed before validation and let $\oplus$ be deterministic. Let $\hat{U}_{k}(A)$ , computed once for each agent $A$ , be the mean of $u_{k}$ over $n=n_{k}^{\mathrm{val}}$ tasks drawn independently from $\mathcal{Q}_{k}$ , independent of the items and of $A_{k}^{\mathrm{loc}}$ , with one trajectory sampled from $p_{A}$ for each task. With $e_{k}$ as in Eq. (31), the gate of Eq. (30) yields $\mathrm{CG}_{k}\geq 0$ with probability at least $1-\eta$ .令条目 z1,…,zm 在验证前固定，令 ⊕ 为确定性。令 U^k(A) 为每个智能体 A 计算一次的均值，即 uku 在从 𝒬k 独立抽取的 n=nkval 个任务上的均值，独立于条目和 Akloc，每个任务从 pA 中采样一条轨迹。在式 (31) 中使用 ek，式 (30) 的门控以至少 1−η 的概率产生 CGk≥0。

Proof.

The gate processes the items in order, so it can only reach agents of the form $A_{k}^{\mathrm{loc}}\oplus z_{i_{1}}\oplus\dots\oplus z_{i_{s}}$ with $i_{1}<\dots<i_{s}$ , one for each subset of the items. Let $\mathcal{C}$ be this set of at most $2^{m}$ agents. For a fixed $A\in\mathcal{C}$ , $\hat{U}_{k}(A)$ is the mean of $n$ independent variables in $[0,1]$ with mean $U_{k}(A)$ , so Hoeffding’s inequality gives门控按顺序处理条目，因此它只能到达 Akloc⊕zi1⊕⋯⊕zis 形式的智能体，其中 i1<⋯<is，每个子集对应一个。令 𝒞 为此最多 2m 个智能体的集合。对于固定的 A∈𝒞，U^k(A) 是 [0,1] 中 n 个独立变量的均值，均值为 Uk(A)，因此霍夫丁不等式给出了

\Pr\big[\,|\hat{U}_{k}(A)-U_{k}(A)|>e_{k}\big]\ \leq\ 2\exp\!\big(-2ne_{k}^{2}\big)\ =\ 2\cdot 2^{-(m+1)}\,\eta\ =\ 2^{-m}\,\eta.

A union bound over $\mathcal{C}$ shows that the event $E=\{|\hat{U}_{k}(A)-U_{k}(A)|\leq e_{k}\ \forall A\in\mathcal{C}\}$ has probability at least $1-\eta$ . On $E$ , every accepted step satisfies对 𝒞 进行联合界（union bound）分析表明，事件 E={|U^k(A)−Uk(A)|≤ek∀A∈𝒞} 的概率至少为 1−η。在事件 E 成立的情况下，每一步被接受的步骤都满足

U_{k}\big(A_{k}^{(i)}\big)-U_{k}\big(A_{k}^{(i-1)}\big)\ \geq\ \hat{U}_{k}\big(A_{k}^{(i)}\big)-\hat{U}_{k}\big(A_{k}^{(i-1)}\big)-2e_{k}\ \geq\ 0,

and every rejected step leaves the agent unchanged. Summing over the steps gives $\mathrm{CG}_{k}=U_{k}(A_{k}^{+})-U_{k}(A_{k}^{\mathrm{loc}})=\sum_{i=1}^{m}\big[U_{k}(A_{k}^{(i)})-U_{k}(A_{k}^{(i-1)})\big]\geq 0$ on $E$ . ∎且每一步被拒绝的步骤都不会改变智能体的状态。对所有步骤求和可得 CGk=Uk(Ak+)−Uk(Akloc)=∑i=1m[Uk(Ak(i))−Uk(Ak(i−1))]≥0，该式在事件 E 上成立。∎

Federated Agent Optimization联邦智能体优化

Abstract摘要

1. Introduction1. 引言

Paper Roadmap.论文路线图。

2. Related Work2. 相关工作

Federated Learning and Federated Large Models联邦学习与联邦大模型

Multi-Agent Systems and Agent Reinforcement learning多智能体系统与智能体强化学习

Privacy-Preserving AI隐私保护 AI

3. Problem Definition: Federated Agent Optimization3. 问题定义：联邦智能体优化

3.1 Local Agents and Private Data3.1 本地智能体与私有数据

3.2 Federated Protocol3.2 联邦协议

3.3 Multi-Objective Formulation3.3 多目标公式

Definition 1 (Federated Agent Optimization).定义 1（联邦智能体优化）。

3.4 Relation to Existing Paradigms3.4 与现有范式的关系

4. Taxonomy of Federated Agent Optimization4. 联邦智能体优化分类法

Policy (Δπ\Delta^{\pi}).策略 (Δ^π)。

Memory (Δℳ\Delta^{\mathcal{M}}).记忆 (Δ^ℳ)。

Tool use (Δ𝒯\Delta^{\mathcal{T}}).工具使用 (Δ^𝒯)。

Reward (Δℛ\Delta^{\mathcal{R}}).奖励 (Δ^ℛ)。

Structured knowledge and skills (Δ𝒪\Delta^{\mathcal{O}}).结构化知识和技能 (Δ^𝒪)。

Comparison.比较。

5. How Agents Mutually Improve Each Other5. 智能体如何相互改进

5.1 The Improvement Loop5.1 改进循环

5.2 Five Types of Shared Knowledge5.2 五种共享知识类型

5.3 When Is the Improvement Mutual?5.3 改进何时是相互的？

6. Key Challenges6. 关键挑战

6.1 Performance, Privacy and Communication Trade-offs6.1 性能、隐私和通信的权衡

6.2 How to Ensure Fairness in Federated Agent Systems?6.2 如何确保联邦智能体系统中的公平性？

6.3 How to Evaluate Federated Agent Systems?6.3 如何评估联邦智能体系统？

7. Future Works7. 未来工作

8. Conclusion8. 结论

References参考文献

Appendix A Notation附录 A 符号说明

Appendix B Methods that Federate a Component of an Agent附录 B 联邦化智能体组件的方法

B.1 Federated Policy OptimizationB.1 联邦策略优化

Weights.权重。

Prompts.提示词。

Trade-offs.权衡。

B.2 Federated Memory OptimizationB.2 联邦记忆优化

Aggregation.聚合。

Trade-offs.

B.3 Federated Tool-Use OptimizationB.3 联邦工具使用优化

Trade-offs.

B.4 Federated Reward OptimizationB.4 联邦奖励优化

Federated preference and reward models.联邦偏好和奖励模型。

Building blocks for agents.智能体的构建块。

Trade-offs.

B.5 Federated Structured Knowledge and Skill OptimizationB.5 联邦结构化知识和技能优化

Ontologies.本体。

Rules.规则。

Skills.技能。

Trade-offs.

B.6 ComparisonB.6 比较

Communication.通信。

Privacy.隐私。

Access to the backbone.骨干模型访问权限。

Maturity.成熟度。

Appendix C Knowledge Types in the Running Example附录 C 运行示例中的知识类型

C.1 The Four Types in the Running ExampleC.1 运行示例中的四种类型

Experience.经验。

Skills.

Rewards.奖励。

Ontologies.

Rules.

Interactions among the types.类型间的相互作用。

C.2 From Shared Knowledge to Local ImprovementC.2 从共享知识到本地改进

Appendix D Extensions of the Formulation附录 D 公式的扩展

Appendix E Proofs附录 E 证明

Proposition 1 (Scalarized forms).命题 1（标量化形式）。

Proof.证明。

Proposition 2 (Gated adaptation).命题 2（门控适应）。

Proof.

Policy ( $\Delta^{\pi}$ ).策略 (Δ^π)。

Memory ( $\Delta^{\mathcal{M}}$ ).记忆 (Δ^ℳ)。

Tool use ( $\Delta^{\mathcal{T}}$ ).工具使用 (Δ^𝒯)。

Reward ( $\Delta^{\mathcal{R}}$ ).奖励 (Δ^ℛ)。

Structured knowledge and skills ( $\Delta^{\mathcal{O}}$ ).结构化知识和技能 (Δ^𝒪)。