Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications人在回路人工智能：概念、方法与应用的系统综述

1.1. Distinct Contributions Relative to Prior HITL Surveys1.1. 相对于以往HITL综述的独特贡献

In order to clarify the novelty of the manuscript with respect to previous HITL review papers, four key distinguishing factors are emphasized. First, an integrative 3D taxonomy is utilized (loop placement, interaction granularity, temporal characteristics), which allows for a more straightforward comparison across systems that may be classified under similar HITL designations but differ with respect to workload, latency tolerance, and oversight costs. Second, a wide range of application domains from healthcare to autonomous systems, cybersecurity, finance, education, and industry is synthesized under a unified framework that allows for cross-domain comparison while respecting domain-specific constraints. Third, whereas previous reviews focus primarily on the description of the various methods available for HITL design, a synthesis of the relevant trust calibration and ethical governance considerations is provided. Finally, a design-focused approach is taken that emphasizes the connections between the various method families, failure modes, and ethical considerations to configuration possibilities within a practical HITL design.为了阐明本文相对于以往HITL综述论文的新颖性，我们强调了四个关键的区分因素。首先，采用了一个综合的三维分类法（回路位置、交互粒度、时间特性），这使得在那些可能被归类为相似HITL名称但在工作量、延迟容忍度和监督成本方面存在差异的系统之间进行比较变得更加直接。其次，在一个统一的框架下综合了从医疗保健到自主系统、网络安全、金融、教育和工业等广泛的应用领域，该框架在尊重特定领域约束的同时，允许跨领域比较。第三，以往的综述主要侧重于对各种HITL设计方法的描述，而本文则提供了对相关信任校准和伦理治理考量的综合。最后，本文采取了一种以设计为中心的方法，强调了各种方法族、故障模式和伦理考量与实际HITL设计中配置可能性之间的联系。

1.2. Systematic Review Protocol1.2. 系统综述协议

To clarify the review methodology, we followed a PRISMA-aligned workflow for evidence identification, screening, eligibility assessment, and synthesis. The core systematic corpus used for structured synthesis comprised 134 studies; additional references were included selectively for methodological background, policy context, or illustrative domain examples.为了阐明综述方法，我们遵循了PRISMA一致的工作流来进行证据识别、筛选、资格评估和综合。用于结构化综合的核心系统语料库包含134项研究；此外，还选择性地纳入了参考文献以提供方法论背景、政策背景或说明性的领域示例。

• Database search strategy.数据库搜索策略。

We searched Scopus and Google Scholar for studies published between January 2018 and January 2026. The search combined HITL and domain terms, including variants of “human-in-the-loop”, “human oversight”, “human-AI collaboration”, “human-on-the-loop”, “active learning”, “RLHF”, “AI governance”, and domain qualifiers (healthcare, autonomous systems, cybersecurity, finance, education, and manufacturing). Forward/backward citation tracking was then applied to capture high-relevance studies not retrieved in the initial database query.我们在Scopus和Google Scholar中搜索了2018年1月至2026年1月期间发表的研究。搜索结合了HITL和领域术语，包括“人在回路”（human-in-the-loop）、“人工监督”（human oversight）、“人机协作”（human-AI collaboration）、“人在环外”（human-on-the-loop）、“主动学习”（active learning）、“RLHF”、“AI治理”（AI governance）的变体，以及领域限定词（医疗保健、自主系统、网络安全、金融、教育和制造业）。随后应用前向/后向引文追踪，以捕获初始数据库查询中未检索到的高相关性研究。

• Inclusion/exclusion criteria.纳入/排除标准。

Studies were included when they: (i) addressed explicit human involvement in AI system training, inference, supervision, or governance; (ii) reported conceptual, methodological, or empirical findings relevant to HITL design/evaluation; and (iii) were available in full text in English. Studies were excluded when they: (i) used HITL terminology without substantive human-role specification; (ii) were purely opinion/editorial pieces without analyzable technical or empirical contribution; (iii) duplicated substantially overlapping content; or (iv) fell outside the scope of AI-assisted decision systems.纳入标准为：(i) 明确涉及人类参与AI系统训练、推理、监督或治理；(ii) 报告了与HITL设计/评估相关的概念、方法或实证研究结果；(iii) 提供英文全文。排除标准为：(i) 使用了HITL术语但未明确人类角色；(ii) 纯粹的观点/评论文章，没有可分析的技术或实证贡献；(iii) 重复了大量重叠的内容；或 (iv) 超出了AI辅助决策系统的范围。

• Screening and selection process.筛选和选择过程。

Screening proceeded in three stages: title/abstract review, full-text eligibility assessment, and final synthesis coding. Figure 2 summarizes the resulting flow.筛选分为三个阶段：标题/摘要审查、全文资格评估和最终综合编码。图2总结了由此产生的流程。

2.1. Anchor Tables: Methods and Domains2.1. 锚点表格：方法与领域

To help readers navigate the HITL design space, Table 1 and Table 2 summarize (i) common HITL method families and (ii) how HITL design choices typically manifest across high-stakes application domains.为了帮助读者导航HITL设计空间，表1和表2总结了 (i) 常见的HITL方法族，以及 (ii) HITL设计选择在高风险应用领域中通常如何体现。

Table 1 organizes method families by required human input, indicative cost, key risks, and common failure modes; bullet-style entries in dense cells are used to improve scanability and support quick cross-row comparison.表1按所需人类输入、指示性成本、关键风险和常见故障模式组织了方法族；密集单元格中的项目符号用于提高可扫描性并支持快速的行间比较。

Table 2 compares application domains using consistent headings (human oversight points, regulation/standards pressure, evaluation metrics, and pitfalls) to make differences in oversight design and deployment constraints easier to interpret.表2使用一致的标题（人工监督点、监管/标准压力、评估指标和陷阱）比较了应用领域，使监督设计和部署约束方面的差异更容易解读。

The intellectual underpinnings of Human-in-the-Loop AI are varied, drawing upon cybernetics, cognitive science, decision theory, as well as human factors engineering. An understanding of the intellectual underpinnings of HITL AI systems provides a context for the design principles and techniques that characterize modern HITL systems. This section follows the historical path of human–machine collaboration, the philosophical underpinnings of the current research, as well as the concept of hybrid intelligence, which structures much of the current research in the field.人在回路人工智能的知识基础多种多样，借鉴了控制论、认知科学、决策理论以及人因工程学。对HITL AI系统知识基础的理解，为表征现代HITL系统的设计原则和技术提供了背景。本节遵循人机协作的历史路径、当前研究的哲学基础，以及混合智能的概念，这些构成了当前该领域大部分研究的结构。

2.2. Historical Evolution2.2. 历史演变

The role of human judgment within an automated system can be traced back to before the advent of modern artificial intelligence by several decades. The foundational work done by cybernetic researchers in the 1940s and 1950s provided foundational principles for feedback control that are still relevant today for human-in-the-loop (HITL) design. The work done by Norbert Wiener on human–machine systems identified the need for mechanisms to be present that enable humans to observe system behavior and act accordingly. These ideas contributed to the development of decision support systems in the 1960s and 1970s, which did not replace human decision-makers but complemented them in business management or military command scenarios [5].自动化系统中人类判断的作用可以追溯到现代人工智能出现之前的几十年。20世纪40年代和50年代控制论研究人员所做的基础工作，为反馈控制提供了至今仍与人在回路（HITL）设计相关的重要原则。Norbert Wiener在人机系统方面的工作确定了需要存在使人类能够观察系统行为并据此采取行动的机制。这些思想促成了20世纪60年代和70年代决策支持系统的发展，这些系统并没有取代人类决策者，而是在企业管理或军事指挥场景中对他们进行了补充 [5]。

The shift from decision support paradigms to interactive machine learning can be seen as a major shift in the way the role of humans is conceptualized in intelligent systems. Instead of seeing humans as mere receivers of system-suggested decisions, interactive approaches emphasize the role of humans in the process of machine learning itself. This shift is also partly driven by the understanding that many problems in reality involve tacit knowledge, contextual information, and value judgments that are difficult to formally specify [10]. The emergence of active learning algorithms in the 1990s formalized methods for systems to query human experts strategically, optimizing the use of scarce human attention and expertise.从决策支持范式向交互式机器学习的转变，可以看作是智能系统中人类角色概念化方式的重大转变。交互式方法不再将人类仅仅视为系统建议决策的接收者，而是强调了人类在机器学习过程本身中的作用。这种转变在一定程度上也是由这样一种认识驱动的：现实中的许多问题涉及难以正式说明的隐性知识、背景信息和价值判断 [10]。20世纪90年代主动学习算法的出现，使系统能够战略性地查询人类专家的方法正式化，从而优化了对稀缺人类注意力和专业知识的使用。

Recent advances in deep learning have paradoxically reinforced the importance of human involvement despite dramatic improvements in automated performance. The opacity of neural network models, their susceptibility to distributional shift, and their potential for encoding harmful biases have made human oversight essential for responsible deployment [11]. The present-day focus of human-in-the-loop (HITL) studies is on both using human knowledge to improve model performance and offering effective human oversight/correction for AI systems that are integrated into complex environments. This is a process of maturity in understanding that we are not trying to minimize human involvement but are instead seeking to maximize its utilization in conjunction with machine performance based on the capabilities of each.深度学习的最新进展矛盾地强化了人类参与的重要性，尽管自动化性能有了显著提高。神经网络模型的不透明性、对分布偏移的敏感性以及编码有害偏见的潜力，使得人工监督对于负责任的部署至关重要 [11]。当今人在回路（HITL）研究的重点既在于利用人类知识来提高模型性能，也在于为集成到复杂环境中的AI系统提供有效的人工监督/纠正。这是一个成熟的过程，我们认识到我们并不是要尽量减少人类的参与，而是寻求在结合机器性能的基础上，根据各自的能力最大限度地利用人类的参与。

2.3. Philosophical and Cognitive Perspectives2.3. 哲学与认知视角

The human-in-the-loop (HITL) design also gives rise to some fundamental issues related to the very nature of intelligence and human judgment. The proponents of human-centric views argue that some aspects of decision-making processes, such as morality and accountability, should remain exclusive to human capabilities and should not be entrusted to machines regardless of their capabilities [12]. This view draws support from phenomenological traditions that highlight the embodied and situated nature of human understanding, which differs qualitatively from computational information processing.人在回路（HITL）设计也引发了一些与智能和人类判断本质相关的基本问题。以人为本观点的支持者认为，决策过程的某些方面，如道德和问责制，应仅限于人类能力，无论机器的能力如何，都不应委托给机器 [12]。这一观点得到了现象学传统的支持，这些传统强调了人类理解的具身性和情境性，这与计算信息处理在性质上是不同的。

The bounded rationality concept, which was first proposed by Herbert A. Simon, provides a different point of view that has major implications for the design of human-in-the-loop systems. According to Simon’s theory, decision-makers are faced with information constraints, cognitive limitations, and time constraints that force them to satisfice rather than optimize. In this regard, human-in-the-loop systems have the ability to deal with vast amounts of information, identify relevant choices, and present information in a format that is cognitively friendly [12]. At the same time, designers must recognize that AI systems introduce their own forms of bounded rationality, including training data limitations, objective function misspecification, and inability to reason about situations outside their training distribution.赫伯特·A·西蒙（Herbert A. Simon）首次提出的有限理性概念，提供了另一个具有重大意义的观点，对人在回路系统的设计产生了深远影响。根据西蒙的理论，决策者面临信息约束、认知局限和时间约束，这迫使他们满足于满意而非最优。在这方面，人在回路系统有能力处理大量信息，识别相关选择，并以认知友好的格式呈现信息 [12]。与此同时，设计者必须认识到AI系统引入了它们自身形式的有限理性，包括训练数据限制、目标函数定义不当以及无法对训练分布之外的情况进行推理。

The question of cognitive load management is central to effective HITL design. Humans interacting with AI systems must process system outputs, maintain situational awareness, make decisions, and provide feedback, all while managing competing demands on attention and working memory [13]. Research in human factors has established that poorly designed automation can actually degrade human performance by inducing complacency, reducing skill maintenance, or overwhelming operators with alerts and information. Effective HITL architectures must therefore balance the benefits of automation against these cognitive costs, designing interactions that keep humans appropriately engaged without exceeding their information processing capacity [14].认知负荷管理的问题对于有效的HITL设计至关重要。与AI系统交互的人类必须处理系统输出、保持态势感知、做出决策并提供反馈，同时还要管理对注意力和工作记忆的相互竞争的需求 [13]。人因研究已经证实，设计糟糕的自动化实际上会通过导致自满、降低技能维持或用警报和信息淹没操作员来降低人类的绩效。因此，有效的HITL架构必须平衡自动化的收益与这些认知成本，设计出既能让人们适当地参与又不会超过其信息处理能力的交互方式 [14]。

2.4. Hybrid and Centaur Intelligence2.4. 混合智能与半人马智能

The metaphor of the centaur, a mythological creature combining human and equine elements, has gained currency as a way of conceptualizing human–AI collaboration. Tang proposed the Chiron Imperative, a framework identifying six models for creating human–AI centaurs that combine the wisdom and ethical judgment of humans with the computational power of AI systems [15]. This framework emphasizes that effective collaboration requires more than simply dividing tasks between humans and machines. Instead, it calls for designing systems where human and artificial intelligence amplify each other’s capabilities in ways that neither could achieve alone.半人马（一种结合了人类和马元素的希腊神话生物）的隐喻，已成为概念化人机协作的一种流行方式。Tang提出了“喀戎命令”（Chiron Imperative），这是一个识别了六种创建人机半人马模型的框架，这些模型结合了人类的智慧和伦理判断与AI系统的计算能力 [15]。该框架强调，有效的协作需要的不仅仅是简单地划分人类和机器的任务。相反，它呼吁设计这样的系统：人类智能和人工智能以双方单独都无法实现的方式相互放大对方的能力。

The idea of the centaur is rooted in the domain of competitive chess, where human–computer collaborations have demonstrated capabilities beyond the capabilities of either humans or computers individually. In the context of freestyle chess tournaments, the winning teams did not comprise the individuals with the highest capabilities or the computers with the highest capabilities, but rather the teams that developed the best methods for interacting with one another. This suggests that the interface for interaction between humans and artificial intelligence systems can be as important as the capabilities of the human or the artificial intelligence system [16]. Translating this insight to other domains requires understanding the specific forms of complementarity that exist between human and machine capabilities in each application context.半人马的思想植根于竞技象棋领域，在那里，人机协作展示了超越人类或计算机单独能力的能力。在自由式象棋锦标赛的背景下，获胜的团队并不是由能力最强的人或能力最强的计算机组成的，而是由开发出最佳交互方法的团队组成的。这表明，人与人工智能系统之间的交互界面可能与人类或人工智能系统的能力同样重要 [16]。将这一见解转化为其他领域，需要理解在每个应用背景下人类和机器能力之间存在的特定互补形式。

The hybrid intelligence concept extends beyond task allocation to consider how human and machine learning can co-evolve over time. In this view, HITL systems are not static configurations but dynamic partnerships where both participants adapt based on their interactions [5]. Human cognition involves creating mental constructs of the capabilities and limitations of artificial intelligence systems and being able to identify when to trust the advice provided by the system or when to trust the decision-making capabilities provided by the artificial intelligence on their own. At the same time, artificial intelligence systems can be designed that mirror human desires, levels of expertise, and decision-making patterns in order to optimize the effectiveness of collaboration between humans and artificial intelligence systems [8].混合智能的概念超越了任务分配，考虑了人类和机器的学习如何随时间共同进化。在这种观点下，HITL系统不是静态的配置，而是动态的伙伴关系，双方都根据彼此的交互进行调整 [5]。人类认知涉及构建对人工智能系统能力和局限性的心理构念，并能够识别何时信任系统提供的建议，或何时信任人工智能自身提供的决策能力。与此同时，可以设计出反映人类欲望、专业水平和决策模式的人工智能系统，以优化人机协作的有效性 [8]。

2.5. Terminology and Loop Configurations2.5. 术语与回路配置

The proliferation of terminology for different human–AI relations is an outcome of both the diversity of the methodological approaches adopted and the lack of standardized terminology within the domain. The most common distinction is between the human-in-the-loop and human-on-the-loop relations. In the former, human participation is necessary for system functioning, often by means of active participation in decisions or authorizing system actions. In the latter case, human monitoring of system functioning is optional while allowing for possible intervention [17].人机关系术语的激增，既是所采用方法论多样性的结果，也是该领域缺乏标准化术语的结果。最常见的区别是“人在回路”（human-in-the-loop）和“人在环外”（human-on-the-loop）关系。在前者中，人类的参与对于系统运行是必要的，通常是通过积极参与决策或授权系统行动的方式。在后者中，人类对系统运行的监控是可选的，同时允许可能的干预 [17]。

Singh and Szajnfarber proposed a more nuanced taxonomy that distinguishes Human-in-the-Loop, Human-on-the-Loop, Human-over-the-Loop, Human-under-the-Loop, and Human-along-the-Loop configurations [17]. Each configuration implies different relationships in terms of power, responsibility, and interaction rate between humans and AI systems. Human-over-the-Loop suggests a configuration where humans are in a position of power with respect to system goals and constraints. Human-under-the-Loop describes a configuration where AI systems are used for controlling or influencing human behavior. Human-along-the-Loop suggests a configuration where humans and AI systems perform related tasks in parallel with lateral interaction. To clarify the operational roles of humans and AI in HITL settings, Table 3 contrasts common loop configurations and their typical deployment contexts.Singh和Szajnfarber提出了一种更细致的分类法，区分了人在回路（Human-in-the-Loop）、人在环外（Human-on-the-Loop）、人在环上（Human-over-the-Loop）、人在环下（Human-under-the-Loop）和人在环旁（Human-along-the-Loop）配置 [17]。每种配置都意味着人类与AI系统之间在权力、责任和交互速率方面存在不同的关系。人在环上（Human-over-the-Loop）暗示了一种人类在系统目标和约束方面处于权力地位的配置。人在环下（Human-under-the-Loop）描述了一种AI系统用于控制或影响人类行为的配置。人在环旁（Human-along-the-Loop）暗示了一种人类和AI系统并行执行相关任务并进行横向交互的配置。为了阐明HITL设置中人类和AI的操作角色，表3对比了常见的回路配置及其典型的部署环境。

The choice among these configurations depends on multiple factors, including the stakes involved in decisions, the reliability of AI components, regulatory requirements, and the availability of qualified human operators [6]. High-stakes applications with high harm potential usually require a Human-in-the-Loop or a Human-over-the-Loop configuration to ensure significant control. Conversely, low-stakes applications may be satisfied with a Human-on-the-Loop monitoring configuration. Understanding the different configuration options and their implications is important to the architects of human–AI systems to achieve a balance between performance, safety, and resource efficiency [18]. Figure 3 illustrates these Human–AI loop configurations, highlighting how different placements of human involvement correspond to varying levels of oversight and autonomy.这些配置之间的选择取决于多种因素，包括决策涉及的风险、AI组件的可靠性、监管要求以及合格人类操作员的可用性 [6]。具有高危害潜力的高风险应用通常需要“人在回路”或“人在环上”配置，以确保足够的控制。相反，低风险应用可能满足于“人在环外”的监控配置。了解不同的配置选项及其影响对于人机系统的架构师来说非常重要，以实现性能、安全性和资源效率之间的平衡 [18]。图3说明了这些人机回路配置，强调了人类参与的不同位置如何对应于不同程度的监督和自主性。

2.6. Conceptual Scope and Term Boundaries2.6. 概念范围与术语边界

In consideration of the tendency for these terms to be used interchangeably in the literature, clear scope distinctions are provided in this review. HITL AI is specified as the overarching design paradigm by which human input has operational impact on model development, deployment, supervision, or governance. Human–AI collaboration is a more general socio-technical construct that includes HITL AI but also spans configurations where humans and AI collaborate without a discernible loop-based control structure. Interactive machine learning (IML) is specified as a methodological sub-set of HITL AI that prioritizes the iterative update of models based on ongoing human interaction. Reinforcement learning from human feedback (RLHF) is specified as a particular type of HITL-based training paradigm that involves the conversion of human preference or critique into reward functions for policy optimization. Explainable AI (XAI) is specified as a supporting layer rather than a type of loop that provides benefits in terms of increased interpretability, trust calibration, or auditability without necessarily providing a basis for meaningful human control.考虑到文献中这些术语倾向于互换使用，本综述提供了明确的范围区分。HITL AI被指定为一种总括性的设计范式，通过该范式，人类输入对模型开发、部署、监督或治理产生操作影响。人机协作是一个更通用的社会技术构造，它包括HITL AI，但也涵盖了人类和AI在没有明显基于回路的控制结构的情况下进行协作的配置。交互式机器学习（IML）被指定为HITL AI的一个方法论子集，优先考虑基于持续人类交互的模型迭代更新。基于人类反馈的强化学习（RLHF）被指定为一种特定类型的基于HITL的训练范式，涉及将人类偏好或批评转化为政策优化的奖励函数。可解释AI（XAI）被指定为一种支持层，而不是一种回路类型，它在提高可解释性、信任校准或可审计性方面提供好处，而不一定为有意义的人类控制提供基础。

Consequently, the definitional hierarchy that is used in this survey is: (i) HITL AI as the overarching framework, (ii) method families such as IML and RLHF as specific instantiations of that framework, and (iii) cross-cutting enablers such as XAI and trust calibration that enable effective oversight of multiple methods and domains. This hierarchy is used uniformly in the following sections of this survey to identify: (i) the loop structure, (ii) the technical integration of human feedback, and (iii) the enablers that make oversight effective in practice.因此，本综述中使用的定义层级为：(i) HITL AI作为总括框架，(ii) IML和RLHF等方法族作为该框架的具体实例化，以及 (iii) XAI和信任校准等跨领域赋能因素，它们使多种方法和领域的有效监督成为可能。该层级在下文中被统一使用，以识别：(i) 回路结构，(ii) 人类反馈的技术整合，以及 (iii) 使监督在实践中有效的赋能因素。

2.7. Interaction Granularity and Temporal Characteristics2.7. 交互粒度与时间特性

While loop placement describes where human authority is positioned, two additional dimensions are needed to characterize how collaboration unfolds in practice: interaction granularity and temporal characteristics. These dimensions affect annotation cost, cognitive load, latency, and safety, and therefore influence whether a HITL design remains feasible at deployment scale [13,18].虽然回路位置描述了人类权威的定位，但还需要另外两个维度来表征协作在实践中如何展开：交互粒度和时间特性。这些维度影响标注成本、认知负荷、延迟和安全性，因此影响HITL设计在部署规模上是否仍然可行 [13,18]。

The concept of interaction granularity refers to the level and detail of human input that is expected by the system. The coarse-grained interaction model describes human input as being sparse and high-level. Examples include approval or rejection of a model’s output, escalation decisions, and quality assessment at a batch level. The medium-grained interaction model includes selective corrections, ranking or alternatives, and labeling of uncertain data in active learning. The fine-grained interaction model requires detailed human input. Examples include token-level corrections, corrections in feature attributions, trajectory guidance in reinforcement learning, and step-wise guidance in interactive generation. The more fine-grained the interaction model, the greater the possibility for precise matching and the greater the human input burden and fatigue [19,20].交互粒度的概念是指系统所期望的人类输入的水平和细节。粗粒度交互模型将人类输入描述为稀疏且高级的。示例包括对模型输出的批准或拒绝、升级决策以及批处理级别的质量评估。中粒度交互模型包括选择性更正、排序或替代方案，以及主动学习中不确定数据的标注。细粒度交互模型需要详细的人类输入。示例包括标记级别的更正、特征归因中的更正、强化学习中的轨迹引导以及交互式生成中的逐步引导。交互模型越细粒度，精确匹配的可能性就越大，人类输入的负担和疲劳感也越大 [19,20]。

For example, a coarse-grained interaction may be defined as a physician simply accepting or rejecting the model’s suggested level of urgency for a given case under AI-assisted emergency triage. On the other hand, a fine-grained interaction may require the physician to edit specific components of the model’s rationale, such as the weighting of symptoms or risk factors, providing a more detailed corrective signal at a higher temporal and cognitive cost.例如，粗粒度交互可以定义为医生在AI辅助紧急分诊中，仅接受或拒绝模型对给定病例建议的紧急程度。另一方面，细粒度交互可能要求医生编辑模型理由的具体组成部分，例如症状或风险因素的权重，从而以更高的时间和认知成本提供更详细的纠正信号。

The temporal properties concern the rate at which human input is integrated into the model’s behavior. Synchronous interaction requires immediate human involvement in the decision-making process (e.g., confirmation of a clinical decision prior to taking an action), while asynchronous interaction allows delayed input to be used to inform future model behavior (e.g., periodic relabeling or retrospective audit feedback). Another important distinction lies along the dimension of the rate of updates. Continuous feedback flows provide rapid adaptability to changing conditions but may cause unstable model behavior given noisy feedback, while episodic feedback occurs on a pre-scheduled review cycle to provide improved traceability of governance interventions at the expense of response speed [5,8].时间属性关注的是人类输入被整合到模型行为中的速率。同步交互要求在决策过程中立即进行人类参与（例如，在采取行动之前对临床决策进行确认），而异步交互允许将延迟的输入用于为未来的模型行为提供信息（例如，定期重新标注或回顾性审计反馈）。另一个重要的区别在于更新速率维度。持续的反馈流为应对不断变化的条件提供了快速的适应性，但鉴于噪声反馈，可能会导致不稳定的模型行为；而情景性反馈发生在预定的审查周期内，以提供改进的治理干预可追溯性，但代价是响应速度 [5,8]。

There is a strong interaction between these two dimensions and the placement of the loop. For example, human-on-the-loop supervision is often used with coarse-grained and asynchronous interaction for moderate-risk operations, whereas human-in-the-loop control for high-risk operations often requires finer-grained and more synchronous interaction. Likewise, human-over-the-loop governance can use episodic temporal patterns, even for highly automated operation. This overlooks the fact that the placement of the loop is not the only relevant axis for the taxonomy.这两个维度与回路位置之间存在很强的相互作用。例如，人在环外（human-on-the-loop）监督通常用于中等风险操作的粗粒度异步交互，而高风险操作的人在回路（human-in-the-loop）控制通常需要更细粒度、更同步的交互。同样，人在环上（human-over-the-loop）治理甚至可以在高度自动化的操作中使用情景性时间模式。这忽略了一个事实，即回路的位置并不是分类法中唯一相关的轴。

From a design perspective, this three-dimensional taxonomy supports explicit trade-off analysis. Systems that prioritize throughput may select coarser interaction and episodic review, then add targeted synchronous checkpoints for edge cases. Systems that prioritize accountability and value alignment may adopt finer-grained interventions at selected stages while constraining interaction frequency to preserve human attention. Throughout this survey, technical methods (Section 3), application deployments (Section 5), and governance mechanisms (Section 7) are interpreted through these trade-offs to clarify why similar HITL labels can correspond to very different operational realities.从设计角度来看，这种三维分类法支持明确的权衡分析。优先考虑吞吐量的系统可以选择更粗的交互和情景审查，然后为边缘案例添加有针对性的同步检查点。优先考虑问责制和价值对齐的系统可以在选定的阶段采取细粒度干预，同时限制交互频率以保持人类注意力。在本综述中，技术方法（第3节）、应用部署（第5节）和治理机制（第7节）都是通过这些权衡来解读的，以阐明为什么相似的HITL标签可能对应于截然不同的操作现实。

3.1. Cross-Method Comparative Analysis and Applicability Boundaries3.1. 跨方法比较分析与适用性边界

To enable a rich horizontal comparison between methodological classes, this paper considers the primary human-in-the-loop (HITL) techniques along six different dimensions: typical use cases, type of human involvement, interaction costs, scalability, real-time capabilities, and robustness against distribution shift/adversarial pressure. Active learning and human annotation tend to be advantageous in data-scarce domains where label quality is the primary bottleneck; these techniques offer high controllability but can become prohibitively expensive in large-scale annotation scenarios. Reinforcement learning from human feedback and preference optimization can be effective in supporting behavior alignment in generative models but suffer from increased interaction costs and require stronger countermeasures against reward hacking, preference drift, and evaluator inconsistency.为了在方法论类别之间进行丰富的横向比较，本文从六个不同维度考虑了主要的人在回路（HITL）技术：典型用例、人类参与类型、交互成本、可扩展性、实时能力以及针对分布偏移/对抗性压力的鲁棒性。主动学习和人工标注在数据稀缺且标签质量是主要瓶颈的领域往往具有优势；这些技术提供了高可控性，但在大规模标注场景中可能变得极其昂贵。基于人类反馈的强化学习和偏好优化在支持生成模型中的行为对齐方面可能有效，但交互成本增加，并且需要针对奖励黑客攻击、偏好漂移和评估者不一致采取更强的对策。

The interactive machine learning model and human guidance are best suited when iteration is possible during development, where domain specialists are readily available, allowing for rapid adaptation of local task alignment, but they are also prone to non-stationary feedback and operator variance. The post hoc validation and escalation approaches are best suited for scalability and ease of deployment, particularly when uncertainty triage is effective, but are not effective in addressing model structure issues if data or model problems are not addressed. The prompt-based human-in-the-loop model is best suited for cost-effective, rapid adaptation for generative problems but is also prone to brittleness when faced with adversarial examples and needs verification protocols for maintaining factual accuracy.交互式机器学习模型和人类引导最适合于开发过程中可以进行迭代的情况，此时领域专家随时可用，允许快速适应局部任务对齐，但也容易受到非平稳反馈和操作员差异的影响。事后验证和升级方法最适合于可扩展性和易于部署，特别是在不确定性分诊有效时，但如果未解决数据或模型问题，则无法有效解决模型结构问题。基于提示的人在回路模型最适合于生成式问题中具有成本效益的快速适应，但在面对对抗性示例时也容易变得脆弱，并且需要验证协议来维持事实准确性。

In all these dimensions, the fundamental trade-off for each method concerns not just the accuracy of the model itself, but the attention that humans have available over a period of time. Methods that demand a high level of human interaction are those that improve the accuracy of alignment while sacrificing scalability; those that require little or no human interaction heavily depend on the quality of confidence calibration, escalation, and governance. This comparative analysis is used throughout this section to highlight the strength that each method exhibits, the scope for which each method is applicable, and the measures that need to be taken for each method.在所有这些维度中，每种方法的基本权衡不仅涉及模型本身的准确性，还涉及人类在一段时间内可用的注意力。需要高水平人类交互的方法是那些在牺牲可扩展性的同时提高对齐准确性的方法；那些几乎不需要或不需要人类交互的方法则严重依赖于置信度校准、升级和治理的质量。本节全文使用了这种比较分析，以突出每种方法所表现出的优势、每种方法的适用范围，以及每种方法需要采取的措施。

3.2. Active Learning and Human Annotation3.2. 主动学习与人工标注

Active learning represents one of the most mature and widely deployed approaches to Human-in-the-Loop machine learning. The fundamental principle underlying active learning is that machine learning algorithms can achieve better performance with fewer training examples if they are allowed to select the data from which they learn [19]. Rather than training on randomly sampled data, active learning systems identify instances where human annotation would be most informative for improving model performance. This selective approach to data labeling addresses a persistent challenge in machine learning: the high cost and limited availability of human-annotated training data.主动学习是“人在回路”（Human-in-the-Loop）机器学习中最成熟且应用最广泛的方法之一。主动学习的基本原则是，如果允许机器学习算法自主选择学习数据，它们就能以更少的训练样本实现更好的性能 [19]。主动学习系统并非在随机采样的数据上进行训练，而是识别出哪些样本的标注对提升模型性能最有价值。这种选择性的数据标注方法解决了机器学习中一个长期存在的挑战：人工标注训练数据的高昂成本和有限的可用性。

The query strategies employed in active learning systems determine which instances are presented to human annotators. Uncertainty sampling, one of the most common strategies, selects instances for which the current model has the least confidence in its predictions [19]. Query by committee methods utilize a committee of models and select data points based on the strongest disagreement among the committee. Expected model change methods select data points that will result in the largest change in the current model. Huang et al. proposed a fast active learning method that optimizes the selection process for active learning while minimizing computational cost. This demonstrates that a well-designed algorithm can greatly improve the usability of active learning in constrained environments [21]. Each strategy embodies different assumptions about what makes an instance informative, and the choice among them depends on the specific characteristics of the learning task and the available computational resources.主动学习系统中采用的查询策略决定了哪些样本会被呈现给人工标注员。不确定性采样（Uncertainty sampling）是最常见的策略之一，它选择当前模型对其预测最缺乏信心的样本 [19]。委员会查询（Query by committee）方法利用一个模型委员会，根据委员会成员之间分歧最大的数据点进行选择。预期模型变化（Expected model change）方法则选择那些会导致当前模型发生最大变化的数据点。Huang 等人提出了一种快速主动学习方法，在优化主动学习选择过程的同时最大限度地降低了计算成本。这表明，设计精良的算法可以极大地改善主动学习在受限环境下的可用性 [21]。每种策略都包含关于“何为信息量大的样本”的不同假设，具体的选择取决于学习任务的特性以及可用的计算资源。

The design of annotation interfaces and workflows significantly affects both the quality and efficiency of human labeling efforts. Effective annotation systems must balance the need for detailed, accurate labels against the cognitive demands placed on human annotators [34]. Research has shown that annotation quality can degrade substantially when annotators experience fatigue or when task complexity exceeds their working memory capacity. Modern annotation platforms therefore incorporate features such as adaptive task difficulty, real-time feedback on annotation consistency, and mechanisms for identifying and resolving disagreements among multiple annotators. Alla proposed an intelligent automation framework that integrates active learning with AI-driven feedback loops, enabling systems to adapt their query strategies based on annotator performance patterns [22].标注界面和工作流的设计对人工标注工作的质量和效率都有显著影响。有效的标注系统必须在对详细、准确标注的需求与人工标注员的认知负担之间取得平衡 [34]。研究表明，当标注员感到疲劳或任务复杂度超过其工作记忆容量时，标注质量会大幅下降。因此，现代标注平台集成了诸如自适应任务难度、标注一致性的实时反馈，以及识别和解决多名标注员之间分歧的机制等功能。Alla 提出了一种智能自动化框架，将主动学习与人工智能驱动的反馈回路相结合，使系统能够根据标注员的表现模式调整查询策略 [22]。

The emergence of crowdsourcing platforms has expanded the scale at which human annotation can be performed while introducing new challenges related to annotator expertise and quality control. Crowdsourced annotation enables rapid collection of large labeled datasets but requires careful attention to annotator selection, training, and quality assurance [24]. Techniques such as the use of gold standard questions, inter-annotator agreement measures, and weighted aggregation of multiple annotations are helpful in maintaining the quality of labels. Wiethof et al. studied the gamification approach to boost the motivation of the annotators. Gamification can increase the quality of each annotation as well as the overall quality of all annotations by reducing the monotony of the tasks [25]. The trade-offs between expert annotation and crowdsourced labeling depend on task complexity, the availability of domain expertise, and the acceptable level of label noise for the downstream application.众包平台的出现扩大了人工标注的规模，同时也带来了与标注员专业知识和质量控制相关的新挑战。众包标注能够快速收集大规模标注数据集，但需要对标注员的选择、培训和质量保证给予密切关注 [24]。诸如使用黄金标准问题、标注员间一致性度量以及多重标注的加权聚合等技术，有助于保持标注质量。Wiethof 等人研究了通过游戏化手段来提高标注员的积极性。游戏化可以通过减少任务的单调性来提高单次标注的质量，以及所有标注的整体质量 [25]。专家标注与众包标注之间的权衡取决于任务复杂度、领域专业知识的可用性，以及下游应用所能接受的标签噪声水平。

A more challenging case arises when the disagreement is due to underlying ambiguity, insufficiently defined task definitions, or genuine, albeit varying, expert judgment, as opposed to error due to randomness. In these situations, majority voting can lead to the suppression of relevant, albeit minority, interpretations. Human-in-the-loop (HITL) systems can take advantage of the ability to model the annotators, for instance, through probabilistic voting, which can estimate annotator reliability and bias, or confusion matrix-based methods, which can distinguish systematic from random error. Uncertainty can be propagated downstream as soft labels or label distributions, rather than as single, discrete class targets. Operationally, the system can identify the high-disagreement instances as candidates for additional processing, for instance, through additional expert review, while low-disagreement, stable instances are left within the high-throughput annotation system. This addresses the issue of disagreement as a means to improve both the calibration of the model as well as the governance of the system, highlighting the areas of strong human agreement as well as areas of human judgment that are contestable [24,26,35].当分歧并非由随机误差引起，而是源于潜在的歧义、任务定义不充分或确实存在的（尽管各不相同的）专家判断时，情况会更具挑战性。在这种情况下，多数投票可能会导致相关的（尽管是少数的）解释被抑制。人在回路（HITL）系统可以利用对标注员进行建模的能力，例如通过概率投票来估计标注员的可靠性和偏差，或者利用基于混淆矩阵的方法来区分系统性误差与随机误差。不确定性可以在下游作为软标签或标签分布进行传播，而不是作为单一的离散类别目标。在操作层面，系统可以将高分歧样本识别为需要额外处理的候选对象（例如通过额外的专家审核），而将低分歧、稳定的样本留在高吞吐量的标注系统中。这解决了将分歧作为改进模型校准和系统治理手段的问题，突显了人类意见高度一致的领域以及人类判断存在争议的领域 [24,26,35]。

Domain-expert annotation presents distinct challenges and opportunities compared to crowdsourced approaches. In fields such as medical imaging, legal document analysis, and scientific research, annotations require specialized knowledge that cannot be readily obtained from general crowdsourcing platforms [26]. Expert annotators can provide richer, more nuanced labels but are scarce and expensive resources. Chandler et al. examined human-in-the-loop methodologies for psychiatric applications, demonstrating how expert clinicians can be effectively integrated into machine learning workflows while respecting the constraints on their time and cognitive resources [27]. Active learning becomes particularly valuable in expert annotation contexts because it maximizes the information gained from each expert interaction. Hybrid approaches that combine expert annotation for difficult cases with crowdsourced annotation for straightforward instances can achieve favorable trade-offs between cost and quality.与众包方法相比，领域专家标注带来了独特的挑战和机遇。在医学影像、法律文档分析和科学研究等领域，标注需要无法从通用众包平台轻易获取的专业知识 [26]。专家标注员可以提供更丰富、更细致的标签，但他们是稀缺且昂贵的资源。Chandler 等人研究了用于精神病学应用的人人回路方法，展示了如何在尊重专家时间和认知资源限制的前提下，将临床专家有效地整合到机器学习工作流中 [27]。主动学习在专家标注背景下显得尤为宝贵，因为它最大化了从每次专家互动中获得的信息量。结合针对困难案例的专家标注和针对简单实例的众包标注的混合方法，可以在成本和质量之间实现良好的权衡。

Specialized annotation tasks often require custom interfaces and protocols tailored to the specific characteristics of the data and the expertise of annotators. Butler et al. developed a human-in-the-loop system for analyzing facial expression labels, addressing the particular challenges of annotating affective data where ground truth is inherently subjective and context-dependent [36]. Their work illustrates how annotation systems must be designed with careful attention to the nature of the labeling task and the cognitive processes involved in human judgment. Similarly, applications in industrial quality inspection have required annotation interfaces that present visual information in ways that support rapid and accurate defect identification by trained inspectors [23].专业标注任务通常需要针对数据特性和标注员专业知识量身定制的自定义界面和协议。Butler 等人开发了一种用于分析面部表情标签的人人回路系统，解决了在标注情感数据时面临的特殊挑战，即基本事实本质上是主观且依赖于上下文的 [36]。他们的工作阐明了标注系统的设计必须仔细考虑标注任务的性质以及人类判断所涉及的认知过程。同样，工业质量检测的应用也需要标注界面，以支持受过培训的检查员快速、准确地识别缺陷的方式呈现视觉信息 [23]。

Recent advances in generative AI have created new possibilities for human-in-the-loop annotation workflows. Large language models can generate candidate annotations or explanations that human annotators then verify, correct, or refine [34]. This approach can substantially accelerate annotation throughput while maintaining human oversight of the final labels. Chen et al. demonstrated this framework in an educational context, developing a generative AI-based system for creating teaching materials where human educators review and refine AI-generated content [37]. The human function changes from producing annotations de novo to evaluating and editing the proposals that machines make. The empirical results show that a verification-based process can reduce the time spent in annotation while preserving or improving the quality of the annotations, provided that annotators are aware of the potential for automation bias and do not over-rely on machine recommendations.生成式人工智能的最新进展为人在回路的标注工作流创造了新的可能性。大型语言模型可以生成候选标注或解释，然后由人工标注员进行验证、纠正或优化 [34]。这种方法可以在保持人类对最终标签监督的同时，大幅加快标注吞吐量。Chen 等人在教育背景下演示了这一框架，开发了一种基于生成式人工智能的教学材料创建系统，由人类教育工作者审查并优化人工智能生成的内容 [37]。人类的功能从从零开始进行标注转变为评估和编辑机器提出的建议。实证结果表明，基于验证的过程可以减少标注所花费的时间，同时保持或提高标注质量，前提是标注员意识到潜在的自动化偏差，并且不过度依赖机器建议。

The integration of active learning with explainable AI techniques offers promising directions for improving annotation efficiency and quality. When active learning systems can explain why a particular instance was selected for annotation, human annotators gain insight into the model’s current limitations and can provide more targeted feedback [23]. Explanations can also help annotators understand edge cases and ambiguous instances, leading to more consistent labeling decisions. Harris demonstrated how combining human-in-the-loop systems with AI fairness toolkits can help identify and mitigate biases in training data, particularly in sensitive applications such as job hiring algorithms where annotation decisions can have significant social consequences [38]. This combination of active selection, explanatory context, and fairness awareness represents a more sophisticated form of human–AI collaboration than traditional active learning approaches that treat annotation as a simple labeling task.将主动学习与可解释人工智能技术相结合，为提高标注效率和质量提供了有前景的方向。当主动学习系统能够解释为何选择特定样本进行标注时，人工标注员就能深入了解模型当前的局限性，并提供更有针对性的反馈 [23]。解释还可以帮助标注员理解边缘案例和模糊实例，从而做出更一致的标注决策。Harris 演示了将人在回路系统与人工智能公平性工具包相结合，如何有助于识别和减轻训练数据中的偏差，特别是在招聘算法等敏感应用中，标注决策可能产生重大的社会后果 [38]。这种主动选择、解释性背景和公平意识的结合，代表了一种比传统主动学习方法（将标注视为简单的标签任务）更复杂的人机协作形式。

3.3. Human-in-the-Loop Reinforcement Learning3.3. 人在回路强化学习

Reinforcement learning from human feedback has emerged as a powerful approach for training AI systems that align with human preferences and values. Traditional reinforcement learning relies on reward functions that specify desired behavior through numerical signals, but designing appropriate reward functions for complex tasks proves extremely difficult in practice [29]. Human-in-the-loop reinforcement learning addresses this challenge by incorporating human judgment directly into the learning process, either through explicit reward signals, demonstrations of desired behavior, or comparative preferences between alternative actions.基于人类反馈的强化学习已成为训练符合人类偏好和价值的人工智能系统的一种强大方法。传统的强化学习依赖于通过数值信号指定期望行为的奖励函数，但在实践中，为复杂任务设计合适的奖励函数极其困难 [29]。人在回路强化学习通过将人类判断直接纳入学习过程来解决这一挑战，具体方式包括显式的奖励信号、期望行为的演示，或对替代行动的比较偏好。

The most direct form of human involvement in reinforcement learning is human reward shaping, where human observers provide reward signals based on their evaluation of agent behavior. This approach has proven effective in domains where the objectives are clear to human observers but difficult to formalize mathematically [28]. In the context of autonomous driving scenarios, it is easy for humans to ascertain whether a driving action is safe and comfortable or not, even though it would be extremely challenging to define the safety and comfort criteria precisely within a reward function. The role of human rewards is to enable the learning of behaviors that are aligned with what can be considered implicit expectations.强化学习中人类参与最直接的形式是人类奖励塑造（Human reward shaping），即人类观察者根据对智能体行为的评估提供奖励信号。这种方法在目标对人类观察者明确但难以数学形式化的领域已被证明是有效的 [28]。在自动驾驶场景中，人类很容易判断驾驶行为是否安全舒适，尽管要在奖励函数中精确定义安全和舒适标准极具挑战性。人类奖励的作用是实现对符合隐性期望的行为的学习。

Demonstration-based learning, also known as learning from demonstration or imitation learning, leverages human expertise by training agents to replicate observed human behavior. In this context, human experts perform tasks while the system records their actions, and the agent learns a policy that reproduces these demonstrated behaviors [29]. The method proves particularly beneficial when the behavior is difficult to articulate but easy to illustrate, a case that often arises in physical manipulation tasks, physical skills like art, and complex decision-making in dynamic environments. The quality of the learned behavior is heavily dependent on the proficiency of the human demonstrators and the number of demonstrations provided.基于演示的学习（也称为从演示中学习或模仿学习）通过训练智能体复制观察到的人类行为来利用人类专业知识。在此背景下，人类专家执行任务，系统记录他们的行动，智能体学习一种能够重现这些演示行为的策略 [29]。当行为难以言表但易于演示时，该方法特别有效，这种情况常出现在物理操作任务、艺术等身体技能以及动态环境中的复杂决策中。所学行为的质量在很大程度上取决于人类演示者的熟练程度以及提供的演示次数。

The idea of humans as mentors for artificial intelligence can be seen as an extension of demonstration-based learning since it enables continuous learning with human mentoring as opposed to only initial mentoring. Huang et al. introduced a framework that enables mentors to correct the behavior of an agent in real time, provide additional demonstrations for complex scenarios, and adjust learning based on performance [28]. This model of mentorship recognizes that effective learning processes are often realized through adaptive guidance that is responsive to the learner’s current capabilities and specific issues that emerge during training processes. The role of a mentor is one that involves less supervisory control than that of teleoperation but offers more feedback than that of demonstration collection.将人类视为人工智能导师的想法可以看作是基于演示的学习的延伸，因为它实现了持续的导师指导，而不仅仅是初始指导。Huang 等人引入了一个框架，使导师能够实时纠正智能体的行为，为复杂场景提供额外演示，并根据表现调整学习 [28]。这种导师模型认识到，有效的学习过程通常是通过响应学习者当前能力和训练过程中出现具体问题的适应性指导来实现的。导师的角色涉及的监督控制比遥操作少，但提供的反馈比单纯的演示收集更多。

Preference-based reinforcement learning represents a particularly influential approach that has enabled significant advances in language model alignment. Rather than providing explicit rewards or demonstrations, humans express preferences between pairs of agent behaviors, indicating which outcome they prefer [30]. Such preference comparison is subsequently used for training a reward model that represents human values, with the learned reward model guiding the agent learning. The preference-based approach relieves human evaluators from cognitive burdens by replacing absolute judgments with relative comparison, which humans are more likely to do uniformly.基于偏好的强化学习代表了一种极具影响力的方法，它在语言模型对齐方面取得了重大进展。人类不是提供显式的奖励或演示，而是在智能体行为对之间表达偏好，指出他们更倾向于哪种结果 [30]。这种偏好比较随后被用于训练代表人类价值的奖励模型，学习到的奖励模型进而指导智能体的学习。基于偏好的方法通过用相对比较代替绝对判断（人类更容易统一地进行相对比较），减轻了人类评估者的认知负担。

The operational risks involved in reinforcement learning from human feedback are significant and must be addressed as first-class design considerations rather than auxiliary caveats. Reward models carry the risk of encoding evaluator bias, discounting minority opinions, and being vulnerable to reward hacking or specification gaming if policies over-optimize proxy reward signals. There are also risks of preference drift over time, brittleness in the face of distribution shift, and safety regressions that are only discovered post-deployment via interactions. These are the reasons for the importance of ongoing auditing, red teaming, and rollback planning in RLHF pipelines, in addition to optimization (see Table 1).基于人类反馈的强化学习所涉及的操作风险是重大的，必须作为首要的设计考量，而不是辅助性的注意事项。奖励模型存在编码评估者偏差、忽视少数群体意见的风险，如果策略过度优化代理奖励信号，还容易受到奖励黑客攻击或规范博弈的影响。此外，还存在随时间推移的偏好漂移、面对分布偏移时的脆弱性，以及仅在部署后通过交互才发现的安全回归风险。这就是为什么在优化之外，持续审计、红队测试和回滚规划在 RLHF 流水线中至关重要的原因（见表 1）。

Another important difference from a technical standpoint is whether reinforcement learning from human feedback (RLHF) is conducted online or offline. Offline RLHF makes use of a dataset, which can improve the reproducibility of the system as well as pre-deployment governance, although the system may not capture rare threats as well as online RLHF. On the other hand, online RLHF can learn from interaction with users, which can improve the system’s ability to correct its own behavior, although the system may become more susceptible to adversarial attacks, feedback loops, as well as rapid policy change without human intervention. Thus, online RLHF is more difficult to integrate with a high-assurance validation approach than offline RLHF [29,30].从技术角度来看，另一个重要的区别是基于人类反馈的强化学习（RLHF）是在线还是离线进行的。离线 RLHF 利用数据集，这可以提高系统的可重复性以及部署前的治理能力，尽管系统可能无法像在线 RLHF 那样很好地捕捉罕见的威胁。另一方面，在线 RLHF 可以从与用户的交互中学习，这可以提高系统纠正自身行为的能力，尽管系统可能变得更容易受到对抗性攻击、反馈回路以及未经人类干预的快速策略变化的影响。因此，在线 RLHF 比离线 RLHF 更难与高保证验证方法集成 [29,30]。

Safety considerations are paramount in human-in-the-loop reinforcement learning, particularly for applications in autonomous systems and robotics. Learning agents may explore dangerous actions during training, and the consequences of unsafe behavior can be severe in physical environments [28]. Human involvement in these environments has several safety-related functions: detection and prevention of potential hazardous actions before execution, provision of corrective feedback upon the occurrence of hazardous actions, and specification of safety constraints that restrict the action set for the agent. The design of human–AI interfaces for safety-critical reinforcement learning agents should allow for prompt human involvement while disturbing the learning process as little as possible.安全考量在人在回路强化学习中至关重要，特别是在自动驾驶系统和机器人应用中。学习智能体在训练期间可能会探索危险动作，而在物理环境中，不安全行为的后果可能是严重的 [28]。人类在这些环境中的参与具有多项与安全相关的功能：在执行前检测并预防潜在的危险动作、在发生危险动作时提供纠正性反馈，以及指定限制智能体动作集的安全约束。用于安全关键型强化学习智能体的人机界面设计应允许人类及时参与，同时尽可能减少对学习过程的干扰。

The application of human-in-the-loop reinforcement learning to autonomous driving has produced substantial research contributions and practical systems. Autonomous vehicles must navigate complex traffic environments while satisfying multiple objectives including safety, efficiency, passenger comfort, and compliance with traffic rules [29]. Human-in-the-loop approaches enable these systems to learn driving behaviors that satisfy human expectations across these multiple dimensions. Real-time human guidance during training can help agents learn appropriate responses to rare but important situations that might be underrepresented in demonstration data or difficult to specify through reward engineering [28]. Ahmad examined the broader question of how human-in-the-loop AI models can support trustworthy autonomous driving systems, emphasizing the importance of maintaining meaningful human oversight even as vehicle automation capabilities increase [39].人在回路强化学习在自动驾驶中的应用已经产生了大量的研究成果和实际系统。自动驾驶车辆必须在复杂的交通环境中导航，同时满足包括安全、效率、乘客舒适度和遵守交通规则在内的多重目标 [29]。人在回路方法使这些系统能够学习在这些多维度上满足人类期望的驾驶行为。训练期间的实时人类指导可以帮助智能体学习如何应对演示数据中可能代表性不足或难以通过奖励工程指定的罕见但重要的情境 [28]。Ahmad 研究了人在回路人工智能模型如何支持可信自动驾驶系统这一更广泛的问题，强调了即使在车辆自动化能力提高的情况下，保持有意义的人类监督依然重要 [39]。

Control room and industrial applications present distinctive requirements for human-in-the-loop reinforcement learning. Operators in process control environments must manage complex systems with multiple interacting variables, competing objectives, and significant consequences for errors [40]. The reinforcement learning agents can assist the operators in suggesting actions, predicting outcomes, or identifying anomalies; yet, the decision-making prerogative lies in the hands of the operators. Research studies in this area have investigated the cognitive states of the operators, which include fatigue, workload, and trust, that affect the effectiveness of human–AI collaboration. Emmanouilidis et al. researched the integration of human-in-the-loop AI systems into production environments, which pinpointed key factors that affect the effectiveness of the integration [41].控制室和工业应用对人在回路强化学习提出了独特的要求。过程控制环境中的操作员必须管理具有多个相互作用变量、相互竞争的目标以及错误后果严重的复杂系统 [40]。强化学习智能体可以在建议行动、预测结果或识别异常方面协助操作员；然而，决策特权仍掌握在操作员手中。该领域的研究调查了影响人机协作有效性的操作员认知状态，包括疲劳、工作负荷和信任。Emmanouilidis 等人研究了将人在回路人工智能系统集成到生产环境中的问题，指出了影响集成有效性的关键因素 [41]。

Apart from industrial control systems, human-in-the-loop reinforcement learning has also found some applications in building management systems and environmental control systems. Liang et al. proposed a human-in-the-loop AI system for HVAC management that meets both efficiency and comfort requirements. This shows that reinforcement learning agents can be used to meet human requirements that differ from person to person [42]. This application illustrates how human feedback can guide learning in domains where objectives are inherently subjective and where automated systems must adapt to diverse user preferences.除了工业控制系统外，人在回路强化学习在建筑管理系统和环境控制系统中也找到了一些应用。Liang 等人提出了一种用于 HVAC 管理的人在回路人工智能系统，该系统同时满足效率和舒适度要求。这表明强化学习智能体可用于满足因人而异的人类需求 [42]。这一应用说明了在目标本质上是主观的且自动化系统必须适应不同用户偏好的领域中，人类反馈如何引导学习。

Adaptive learning systems in education represent another promising application domain for human-in-the-loop reinforcement learning. Tarun et al. explored how generative AI combined with human-in-the-loop feedback can create personalized learning experiences that adapt to individual student needs [43]. In these systems, human educators are used to provide feedback, which is used to guide the AI system. The reinforcement learning framework is used for the refinement of educational interventions, which is informed by the learning outcomes as well as the educators’ feedback.教育领域的自适应学习系统代表了人在回路强化学习的另一个有前景的应用领域。Tarun 等人探讨了生成式人工智能与人在回路反馈相结合如何创造适应个人学生需求的个性化学习体验 [43]。在这些系统中，人类教育工作者提供反馈，用于指导人工智能系统。强化学习框架用于改进教育干预措施，并根据学习成果以及教育工作者的反馈提供信息。

Swarm intelligence approaches offer an alternative example for incorporating human input into collective AI systems. Rosenberg’s work on artificial swarm intelligence demonstrated that groups of humans connected through real-time feedback systems can function as unified intelligent systems that outperform both individual humans and traditional AI approaches on certain tasks [44]. This approach turns the traditional human-in-the-loop concept on its head, as it involves the incorporation of artificial intelligence into collective human processes, as opposed to incorporating humans into artificial intelligence systems. This results in a hybrid swarms concept, which combines human intuition and understanding with machine-based aggregation and coordination.群体智能方法为将人类输入纳入集体人工智能系统提供了另一个例子。Rosenberg 在人工群体智能方面的工作表明，通过实时反馈系统连接的人类群体可以作为一个统一的智能系统运作，在某些任务上优于个体人类和传统人工智能方法 [44]。这种方法颠倒了传统的人在回路概念，因为它涉及将人工智能纳入集体人类过程，而不是将人类纳入人工智能系统。这产生了一个混合群体概念，将人类的直觉和理解与基于机器的聚合和协调相结合。

3.4. Generative AI with Human-in-the-Loop Feedback3.4. 带有人在回路反馈的生成式人工智能

The accelerated development of generative AI systems, with a focus on large language models, has enabled a number of emerging paradigms for human-in-the-loop interaction that are quite different from traditional machine learning approaches. These generative models are capable of creating text, code, images, and other media at a quality that approaches or rivals human levels, yet they require human intervention to ensure that they are correct with respect to user intent, factual correctness, and ethical appropriateness [31]. The human role in generative AI systems encompasses prompt design, output evaluation, iterative refinement, and ongoing monitoring of system behavior across diverse use cases.生成式人工智能系统的加速发展，特别是对大型语言模型的关注，催生了许多新兴的人在回路交互范式，这些范式与传统的机器学习方法截然不同。这些生成模型能够以接近或媲美人类水平的质量创建文本、代码、图像和其他媒体，但它们需要人类干预以确保其在用户意图、事实正确性和伦理适当性方面是正确的 [31]。人类在生成式人工智能系统中的角色包括提示词设计、输出评估、迭代优化以及对不同用例中系统行为的持续监控。

Prompt engineering is a skill that has emerged as a key competence for successful human–AI collaboration with large language models, where the quality and precision of prompts play a significant role in determining their relevance, accuracy, and utility. Ranade et al. showed that rhetorical strategies can be applied in a systematic way for prompt engineering, conceptualizing the interaction between humans and AI as a communicative process for which principles of effective discourse are well established [31]. This perspective reframes prompt engineering from ad hoc experimentation to a principled practice grounded in communication theory. Effective prompts must convey not only the desired task but also relevant context, constraints, output format preferences, and quality criteria.提示词工程（Prompt engineering）是一项已成为大型语言模型实现成功人机协作的关键能力，提示词的质量和精度在决定其相关性、准确性和实用性方面起着重要作用。Ranade 等人表明，修辞策略可以系统地应用于提示词工程，将人与人工智能之间的交互概念化为一个沟通过程，并为此建立了有效的语篇原则 [31]。这一视角将提示词工程从临时实验转变为基于传播理论的原则性实践。有效的提示词不仅必须传达期望的任务，还必须传达相关的背景、约束、输出格式偏好和质量标准。

However, the quality of the response does not address the underlying structural failure modes of the generative model. The issues of hallucination, factual inconsistency across generated responses, and stability with respect to minor changes in the prompts continue to be core technical risks with HITL. The reasons for these risks are that the responses generated are fluent and plausible even when they are incorrect. This makes human over-trust a significant risk in a high-throughput scenario. This essentially means that HITL-style governance must treat generated responses as statements that must be verified rather than as texts that must be rewritten [32,45].然而，响应的质量并不能解决生成模型潜在的结构性故障模式。幻觉、生成响应间的事实不一致以及对提示词微小变化的稳定性问题，仍然是 HITL 的核心技术风险。这些风险的原因在于，即使生成的内容不正确，其表述也是流畅且看似合理的。这使得人类的过度信任在高吞吐量场景中成为一个重大风险。这本质上意味着，HITL 式的治理必须将生成的响应视为需要验证的陈述，而不是需要重写的文本 [32,45]。

In a technically sound human-in-the-loop (HITL) pipeline, generation and verification are kept decoupled as a matter of course. In most cases, standard security measures include retrieval-grounded generation, citation or evidence fields, and consistency checks among various model versions. In addition, in a scaled-up environment, organizations may employ a system of triage, where generated content is categorized into risk levels, with low-risk content possibly subjected to spot checks, whereas high-priority content, such as medical, legal, or financial, may require a more complex system of structured review and sign-off accountability. In a system where verification is not layered in such a manner, it may become a bottleneck, with human verifiers reverting to superficial approval-based behaviors that are not adequate for infrequent yet significant errors [46,47,48].在技术上健全的人在回路（HITL）流水线中，生成和验证通常是解耦的。在大多数情况下，标准安全措施包括检索增强生成、引用或证据字段，以及不同模型版本之间的一致性检查。此外，在规模化环境中，组织可能会采用分流系统，将生成的内容按风险级别分类，低风险内容可能接受抽查，而医疗、法律或金融等高优先级内容可能需要更复杂的结构化审查和签字问责系统。在验证未以这种方式分层的系统中，它可能会成为瓶颈，人类验证者会退回到肤浅的基于批准的行为，这对于不频繁但重大的错误是不够的 [46,47,48]。

The iterative refinement of generative AI outputs represents a distinctive form of human-in-the-loop interaction. Unlike traditional machine learning where human input primarily occurs during training, generative AI systems enable continuous human feedback during inference [32]. There is a capacity for users to judge the content generated, recognize areas for improvement, and offer remedial advice that can guide the subsequent content. This form of dialogue allows human users to guide the content towards the desired form without the need to specify the requirements a priori. The model is more similar to co-editing than supervising, as the human user and the AI system work together to create content through a series of iterations.生成式人工智能输出的迭代优化代表了一种独特的人在回路交互形式。与人类输入主要发生在训练过程中的传统机器学习不同，生成式人工智能系统允许在推理过程中进行持续的人类反馈 [32]。用户有能力判断生成的内容，识别改进领域，并提供可以指导后续内容的补救建议。这种对话形式允许人类用户在无需预先指定需求的情况下，将内容引导至期望的形式。该模型更类似于共同编辑而不是监督，因为人类用户和人工智能系统通过一系列迭代共同创建内容。

Chain-of-thought prompting and related techniques have demonstrated that encouraging language models to articulate intermediate reasoning steps can substantially improve performance on complex tasks. Atkinson extended this approach through chain-of-code prompting, which integrates human validation at key points in multi-step reasoning processes [33]. Human evaluators can authenticate intermediate conclusions, correct inaccuracies in the reasoning process, and provide guidance when the model is uncertain. This nested human-in-the-loop model improves the reliability of processing complex tasks by combining the generative capabilities of language models with human judgment at the point of decision-making. Fu et al. extended the concept of combining language models with human judgment by incorporating non-monotonic logical reasoning, thereby creating assistive AI agents with more robust reasoning capabilities under uncertain situations [49].思维链提示（Chain-of-thought prompting）及相关技术表明，鼓励语言模型阐明中间推理步骤可以显著提高复杂任务的性能。Atkinson 通过代码链提示扩展了这种方法，在多步推理过程的关键点集成了人类验证 [33]。人类评估者可以认证中间结论，纠正推理过程中的不准确之处，并在模型不确定时提供指导。这种嵌套的人在回路模型通过将语言模型的生成能力与决策点上的人类判断相结合，提高了处理复杂任务的可靠性。Fu 等人通过引入非单调逻辑推理，扩展了将语言模型与人类判断相结合的概念，从而在不确定情况下创建了具有更强推理能力的辅助人工智能智能体 [49]。

The application of human-in-the-loop generative AI to professional domains has produced systems that augment expert capabilities while maintaining appropriate oversight. Bui examined the use of generative AI with human oversight for patent law applications, including AI-assisted drafting, prior art search, and multimodal intellectual property protection [50]. These applications require high accuracy and must satisfy strict professional standards, making human validation essential despite the capabilities of underlying AI systems. Yuan et al. developed Alpha-GPT 2.0, a human-in-the-loop system for quantitative investment that combines language model capabilities with human trader expertise to generate and refine investment strategies [51]. In both cases, the human role extends beyond simple approval to include substantive evaluation of AI-generated content against domain-specific criteria.人在回路生成式人工智能在专业领域的应用已经产生了在保持适当监督的同时增强专家能力的系统。Bui 研究了生成式人工智能在专利法应用中的人类监督使用，包括人工智能辅助起草、现有技术搜索和多模态知识产权保护 [50]。这些应用需要高精度且必须满足严格的专业标准，使得尽管底层人工智能系统能力强大，人类验证仍然必不可少。Yuan 等人开发了 Alpha-GPT 2.0，这是一个用于量化投资的人在回路系统，结合了语言模型能力与人类交易员的专业知识来生成和优化投资策略 [51]。在这两种情况下，人类的角色都超越了简单的批准，包括根据特定领域的标准对人工智能生成的内容进行实质性评估。

The healthcare applications of generative AI pose unique challenges in human-in-the-loop design due to the potential impact of erroneous outputs and the need for accountability. Fahad and Huang suggested a framework for continuous validation in healthcare applications of generative AI outputs. They emphasize that human involvement should be an integral part of the workflow and not just at the end stages [32]. Their framework addresses the issue of maintaining diligent human review in the face of the usual high-quality output of AI systems while also acknowledging that occasional errors can have significant consequences in clinical settings. The construction of a robust human review for medical generative AI needs to take into consideration the cognitive burden on clinicians, time constraints, and the need to prevent the degradation of clinical skills that can result from over-reliance on AI.生成式人工智能的医疗应用由于错误输出的潜在影响和对问责制的需求，在人在回路设计中提出了独特的挑战。Fahad 和 Huang 建议为生成式人工智能输出的医疗应用建立一个持续验证框架。他们强调，人类的参与应该是工作流中不可或缺的一部分，而不仅仅是在最后阶段 [32]。他们的框架解决了在人工智能系统通常高质量的输出面前保持勤勉的人类审查的问题，同时也承认偶尔的错误在临床环境中可能产生重大后果。为医疗生成式人工智能构建稳健的人类审查需要考虑临床医生的认知负担、时间限制，以及防止因过度依赖人工智能而导致临床技能退化的需求。

Financial services represent another domain where generative AI is being deployed with human-in-the-loop safeguards. Singh proposed a five-step governance framework for generative AI in banking that operationalizes trust through structured human oversight at multiple stages [48]. The model recognizes that regulatory demands, reputation, and fiduciary duty require a high level of human oversight for AI outputs in a financial setting. Anniciello et al. studied human-in-the-loop generative AI for insurance decision support. The authors created an explainable system that provides justifications for AI recommendations [45].金融服务是另一个部署了人在回路保障措施的生成式人工智能领域。Singh 为银行业生成式人工智能提出了一个五步治理框架，通过多个阶段的结构化人类监督将信任付诸实践 [48]。该模型认识到，监管需求、声誉和受托责任要求在金融环境中对人工智能输出进行高水平的人类监督。Anniciello 等人研究了用于保险决策支持的人在回路生成式人工智能。作者创建了一个可解释系统，为人工智能建议提供理由 [45]。

The balance between the efficiency of automated systems and the effectiveness of human oversight is a key challenge for the deployment of generative artificial intelligence systems. Verma examined if generative AI could be used as a substitute for human-in-the-loop methods in urban design research. He found that although generative AI could speed up some tasks, human judgment was necessary for evaluating the quality and contextual appropriateness of the design [52]. This finding echoes broader concerns about maintaining meaningful human engagement as AI capabilities improve. Effective human-in-the-loop generative AI systems must be designed to keep humans cognitively engaged and capable of identifying AI errors, rather than reducing humans to passive approvers of AI outputs.自动化系统的效率与人类监督的有效性之间的平衡是部署生成式人工智能系统的一个关键挑战。Verma 研究了生成式人工智能是否可以作为城市设计研究中人在回路方法的替代品。他发现，尽管生成式人工智能可以加快某些任务的速度，但人类判断对于评估设计的质量和上下文适当性是必要的 [52]。这一发现呼应了关于随着人工智能能力提高而保持有意义的人类参与的更广泛担忧。有效的人在回路生成式人工智能系统必须设计为让人们在认知上保持参与，并有能力识别人工智能错误，而不是将人类简化为人工智能输出的被动批准者。

Content generation at scale introduces additional considerations for human-in-the-loop workflows. Nuotio investigated the impact of generative AI on journalistic processes, examining how human-in-the-loop approaches can maintain editorial standards while leveraging AI capabilities for content production [46]. Organizational factors that are relevant for successful integration were identified, e.g., clear job definitions, provision of training for human reviewers, and quality assurance approaches that are adapted for AI-based workflows. Kolagani and Vuppala examined related aspects in the context of enterprise customer services, proposing a hybrid approach for balancing efficiency with human oversight for quality maintenance in these services [47].大规模内容生成为人在回路工作流引入了额外的考量。Nuotio 调查了生成式人工智能对新闻过程的影响，研究了人在回路方法如何在利用人工智能能力进行内容生产的同时保持编辑标准 [46]。确定了与成功集成相关的组织因素，例如明确的工作定义、为人类审查者提供培训，以及适应基于人工智能工作流的质量保证方法。Kolagani 和 Vuppala 在企业客户服务背景下研究了相关方面，提出了一种在这些服务中平衡效率与人类监督以维护质量的混合方法 [47]。

3.5. Explainability, Interpretability, and Trust3.5. 可解释性、可理解性与信任

The ability of humans to comprehend, evaluate, and correctly depend on AI systems is significantly dependent on the explainability of AI systems. Explainable AI is a term that comprises various methods that make AI model behavior comprehensible to humans, thus aiding them in decision-making based on when to trust AI suggestions and when not to [4]. Without adequate explainability, human-in-the-loop oversight becomes superficial, as humans cannot meaningfully evaluate outputs they do not understand. The development of explainable AI methods is therefore not merely a technical convenience but a prerequisite for effective human–AI collaboration.人类理解、评估并正确依赖人工智能系统的能力在很大程度上取决于人工智能系统的可解释性。可解释人工智能是一个包含各种方法的术语，这些方法使人工智能模型的行为对人类而言是可理解的，从而在人类基于何时信任人工智能建议以及何时不信任做出决策时提供帮助 [4]。如果没有足够的可解释性，人在回路的监督就会流于表面，因为人类无法有意义地评估他们不理解的输出。因此，开发可解释人工智能方法不仅是技术上的便利，更是有效人机协作的前提。

The difference between interpretability and explainability, although sometimes fuzzy in practice, implies a number of differences regarding the way an AI system can be made understandable. Interpretability is derived from the intrinsic understandability of the system based on structural properties such as decision trees, rule-based systems, or linear models with a reduced number of features. Conversely, explainable AI refers to methods that provide explanations for models that are not intrinsically interpretable, such as deep neural networks [53]. In the study by Assadi & Safaei, interpretable artificial intelligence is discussed in the context of product recommendation systems. This demonstrates that the effectiveness of incorporating human feedback into the loop is increased when users are able to grasp the rationale behind the decision made by the system. Both methods are intended for improving human understanding; however, there are clear distinctions between them.可理解性（Interpretability）与可解释性（Explainability）之间的区别，尽管在实践中有时很模糊，但意味着使人工智能系统可理解的方式存在许多差异。可理解性源于系统基于结构属性（如决策树、基于规则的系统或特征数量减少的线性模型）的内在可理解性。相反，可解释人工智能是指为非内在可理解的模型（如深度神经网络）提供解释的方法 [53]。在 Assadi & Safaei 的研究中，可理解人工智能是在产品推荐系统的背景下讨论的。这表明，当用户能够掌握系统做出决策背后的基本原理时，将人类反馈纳入回路的有效性就会提高。两种方法都旨在改善人类的理解；然而，它们之间存在明显的区别。

Factual explanations describe the features or patterns that led to a particular AI output, while counterfactual explanations describe what would need to change for the output to be different. Ibrahim et al. conducted an algorithm-in-the-loop analysis comparing these explanation types, finding that their effectiveness depends on the decision context and the expertise of human users [54]. Counterfactual explanations proved particularly valuable for helping users understand decision boundaries and identify actionable changes. The choice between explanation types should be guided by the specific needs of human decision-makers and the characteristics of the decisions they face.事实性解释描述了导致特定人工智能输出的特征或模式，而反事实解释描述了为了使输出不同需要改变什么。Ibrahim 等人进行了一项算法在回路分析，比较了这些解释类型，发现它们的有效性取决于决策背景和人类用户的专业知识 [54]。反事实解释对于帮助用户理解决策边界和识别可操作的改变特别有价值。在解释类型之间的选择应以人类决策者的具体需求和他们所面临决策的特征为指导。

Trust calibration represents a critical challenge in human–AI systems where humans must learn to rely appropriately on AI capabilities. Both over-trust and under-trust can compromise system performance: over-trust leads humans to accept AI errors uncritically, while under-trust causes humans to reject valid AI recommendations [20]. In practice, adoption is strongest when users experience consistently calibrated trust because they view the system as both useful and safe enough to incorporate into routine workflows. Tsiakas and Murray-Rust explored how explainable AI can help humans develop appropriate trust by providing insight into AI reasoning processes and limitations. Their work emphasizes that trust should not be unconditional but calibrated to the actual reliability of AI systems across different situations and task types.信任校准代表了人机系统中一个关键的挑战，即人类必须学会适当地依赖人工智能能力。过度信任和信任不足都会损害系统性能：过度信任会导致人类不加批判地接受人工智能错误，而信任不足会导致人类拒绝有效的人工智能建议 [20]。在实践中，当用户体验到持续校准的信任时，采用率最高，因为他们认为该系统既有用又足够安全，可以纳入日常工作流。Tsiakas 和 Murray-Rust 探讨了可解释人工智能如何通过提供对人工智能推理过程和局限性的洞察，帮助人类建立适当的信任。他们的工作强调，信任不应是无条件的，而应根据人工智能系统在不同情况和任务类型下的实际可靠性进行校准。

The cognitive alignment between AI explanations and human mental models significantly affects whether explanations actually improve human decision-making. Explanations that are technically accurate but do not match how humans think about a problem may fail to improve understanding or may even introduce confusion [13]. Kotsiopoulos et al. examined this issue in industrial defect recognition, developing explanations designed to align with the cognitive mechanisms that expert inspectors use when evaluating product quality. Their approach illustrates the importance of user-centered design in explainable AI, where explanation methods must be tailored to the knowledge and reasoning patterns of intended users.人工智能解释与人类心智模型之间的认知对齐显著影响解释是否真正改善了人类决策。在技术上准确但与人类思考问题方式不匹配的解释，可能无法改善理解，甚至可能引入困惑 [13]。Kotsiopoulos 等人在工业缺陷识别中研究了这一问题，开发了旨在与专家检查员在评估产品质量时使用的认知机制相一致的解释。他们的方法说明了以用户为中心的设计在可解释人工智能中的重要性，其中解释方法必须根据目标用户的知识和推理模式进行定制。

The affective dimensions of human–AI interaction influence how explanations are received and whether they achieve their intended effects. Charoenrat developed an affective and explainable AI-driven model for adaptive learning that considers learner emotional states alongside cognitive factors [14]. The current research recognizes that human interactions with AI systems are not just rational in nature, as human responses to AI systems are also subject to emotional responses to the AI system’s behavior, explanations provided, and the interactive nature of the AI system. Explainable AI systems that consider affective factors may help to achieve more effective human–AI collaborations compared to AI systems that are designed based on cognitive models of human users.人机交互的情感维度影响解释的接收方式以及它们是否达到预期效果。Charoenrat 开发了一种用于自适应学习的情感和可解释人工智能驱动模型，该模型在考虑认知因素的同时也考虑了学习者的情绪状态 [14]。当前研究认识到，人类与人工智能系统的交互本质上不仅仅是理性的，因为人类对人工智能系统的反应也受到对人工智能系统行为、所提供解释以及人工智能系统交互性质的情感反应的影响。考虑情感因素的可解释人工智能系统可能比基于人类用户认知模型设计的人工智能系统更有助于实现有效的人机协作。

The practical implementation of explainable AI in human-in-the-loop systems requires careful attention to explanation timing, format, and level of detail. Explanations that interrupt workflow, require excessive cognitive effort to process, or provide irrelevant detail can reduce rather than enhance human performance [4]. Effective explanation interfaces must balance completeness against usability, providing sufficient information for informed decisions without overwhelming users. Research on explanation design has identified principles such as progressive disclosure, where users can access additional detail on demand, and contrastive explanation, where systems highlight differences from typical cases rather than exhaustively describing all features. Table 5 summarizes trust calibration states in human–AI interaction, outlining their defining characteristics, associated risks, and practical interventions for achieving appropriate reliance.在人在回路系统中实际实施可解释人工智能需要仔细注意解释的时机、格式和详细程度。打断工作流、需要过多的认知努力来处理或提供无关细节的解释可能会降低而不是增强人类的表现 [4]。有效的解释界面必须在完整性与可用性之间取得平衡，在不让用户不知所措的情况下为知情决策提供足够的信息。关于解释设计的研究已经确定了诸如渐进式披露（用户可以按需访问额外细节）和对比性解释（系统突出显示与典型案例的区别，而不是详尽描述所有特征）等原则。表 5 总结了人机交互中的信任校准状态，概述了它们的定义特征、相关风险以及实现适当依赖的实际干预措施。

The relationship between explainability and human learning creates opportunities for AI systems that not only support individual decisions but also help humans develop expertise over time. When explanations reveal the patterns and relationships that underlie AI predictions, humans can internalize this knowledge and apply it in situations where AI assistance is unavailable [20]. This role of explainable AI, as part of the educational goals, points to the design strategy that places human learning as a priority alongside prompt decision support. The systems developed for the achievement of these purposes may create more valuable systems for the future, as they improve human capabilities, not reliance on AI systems.可解释性与人类学习之间的关系为人工智能系统创造了机会，这些系统不仅支持个人决策，还帮助人类随着时间的推移发展专业知识。当解释揭示了人工智能预测背后的模式和关系时，人类可以内化这些知识，并将其应用在人工智能辅助不可用的情况下 [20]。可解释人工智能作为教育目标的一部分，指向了将人类学习作为优先事项与提示决策支持并列的设计策略。为实现这些目的而开发的系统可能会为未来创造更有价值的系统，因为它们提高了人类的能力，而不是对人工智能系统的依赖。

3.6. Trust Calibration and Human–AI Interaction Failures3.6. 信任校准与人机交互失败

The effectiveness of human-in-the-loop systems depends fundamentally on whether humans can develop and maintain appropriate levels of trust in AI components. Trust calibration refers to the alignment between a user’s confidence in an AI system and the system’s actual reliability [11]. When trust is well-calibrated, people are generally able to trust AI recommendations in cases where the system performs well, and use their own judgment in cases where the system tends to perform poorly. The challenge in achieving well-calibrated trust lies in the need for people to build accurate mental models of how well AI systems perform in a wide range of cases.人在回路系统的有效性从根本上取决于人类能否在人工智能组件中建立并保持适当水平的信任。信任校准是指用户对人工智能系统的信心与系统实际可靠性之间的一致性 [11]。当信任得到良好校准时，人们通常能够在系统表现良好的情况下信任人工智能建议，并在系统往往表现不佳的情况下使用自己的判断。实现良好校准信任的挑战在于，人们需要建立准确的心智模型，以了解人工智能系统在各种情况下的表现如何。

Over-trust occurs when humans place excessive confidence in AI systems, leading them to accept erroneous outputs without adequate scrutiny. Agudo et al. conducted empirical studies examining how AI errors propagate through human-in-the-loop processes, finding that humans often fail to detect and correct AI mistakes even when they have the knowledge and ability to do so [11]. This is sometimes termed automation bias or even automation complacency, and this is a major risk when AI is used in situations where errors could have serious consequences. The study also revealed that the rate of error detection reduces when individuals adapt to high accuracy levels of AI, which means that AI’s success could be its own failure in terms of requiring human oversight.过度信任发生在人类对人工智能系统过度自信时，导致他们不加充分审查就接受错误的输出。Agudo 等人进行了实证研究，考察了人工智能错误如何在人在回路过程中传播，发现即使人类有知识和能力纠正人工智能的错误，他们也经常无法检测并纠正这些错误 [11]。这有时被称为自动化偏差，甚至自动化自满，当人工智能被用于错误可能产生严重后果的情况下时，这是一个重大风险。研究还表明，当个人适应了人工智能的高准确率水平时，错误检测率会降低，这意味着人工智能的成功可能在其需要人类监督方面成为其自身的失败。

Under-trust presents the opposite problem, where humans discount valid AI recommendations due to skepticism, unfamiliarity, or negative prior experiences. Baroni et al. developed the AI-TAM model to investigate factors affecting user acceptance and collaborative intention in human-in-the-loop applications [56]. In their study, they were able to identify several determinants of trust, such as perceived usefulness, perceived ease of use, and social influence, which clearly shows that trust development involves a rational assessment of system capabilities as well as contextual factors. Under-trust may cause humans to turn away from AI assistance in cases where the performance of AI systems is significantly better than human judgment alone.信任不足（Under-trust）则呈现出相反的问题，即人类由于怀疑、不熟悉或过往的负面经验，而对有效的 AI 建议不予采纳。Baroni 等人开发了 AI-TAM 模型，旨在探究影响“人在回路”（human-in-the-loop）应用中用户接受度与协作意愿的因素 [56]。在他们的研究中，他们识别出了若干信任决定因素，如感知有用性、感知易用性以及社会影响，这清楚地表明，信任的建立既涉及对系统能力的理性评估，也涉及情境因素。在 AI 系统性能明显优于人类单纯判断的情况下，信任不足可能会导致人类拒绝 AI 的辅助。

The dynamics of trust development over extended interaction periods introduce additional complexity. Lopes conducted studies on operator fatigue, trust, and workload demand in human-in-the-loop AI-enabled drone systems, revealing how trust evolves as operators gain experience and as their cognitive resources become depleted [57]. The initial level of trust, whether high or low, has a tendency to set a foundation for future trust evaluations. As such, interactions with AI at the onset have a strong impact. Fatigue was shown to affect the calibration of trust by reducing cognitive resources for monitoring and evaluation. The influence of individual differences on trust calibration has received increasing research attention. Dores Cruz et al. demonstrated that political preferences can compromise human-in-the-loop oversight of AI, with individuals showing systematic biases in how they evaluate AI outputs depending on whether those outputs align with their prior beliefs [58]. This result carries significant implications for applications in which AI systems are involved in discussions about politically or socially contested issues, suggesting that a variety of oversight bodies may be necessary in order to combat individual biases. In a more general sense, the present study underscores that trust in AI systems cannot be accounted for by system-related factors alone, but is shaped by what people believe, value, and cognitively tend toward in their interactions with others.在长时间的交互过程中，信任发展的动态性带来了额外的复杂性。Lopes 对“人在回路”AI 赋能无人机系统中的操作员疲劳、信任度和工作负载需求进行了研究，揭示了随着操作员经验的积累及其认知资源的耗尽，信任是如何演变的 [57]。初始信任水平（无论高低）往往会为未来的信任评估奠定基础。因此，最初与 AI 的交互具有很强的影响力。研究表明，疲劳会通过减少用于监控和评估的认知资源，从而影响信任的校准。个体差异对信任校准的影响已受到越来越多的研究关注。Dores Cruz 等人证明，政治倾向会影响人类对 AI 的监督，个体在评估 AI 输出时会表现出系统性偏见，具体取决于这些输出是否符合他们先前的信念 [58]。这一结果对于 AI 系统参与政治或社会争议议题讨论的应用具有重要意义，表明为了克服个体偏见，可能有必要设立多种监督机构。从更广泛的意义上讲，本研究强调，对 AI 系统的信任不能仅用系统相关因素来解释，它还受到人们在与他人互动时所持有的信念、价值观及认知倾向的影响。

The issue of transparency in regard to the boundaries of artificial intelligence is one of the ways of building calibrated trust, but the relationship between transparency and calibrated trust is not immediately clear. Brooks argues that it is important to maintain proper expectations of artificial intelligence in order to enable effective cooperation between humans and artificial intelligence, but this should not be done in an optimistic or dismissively skeptical way [55]. Communicating uncertainty and limitations can help in the development of accurate mental models in humans; however, over-hedging can create a lack of confidence in the face of valuable assistance from an artificial intelligence system. Transparency that is effective requires calibration in the communication of limitations.关于人工智能边界的透明度问题是建立校准信任（calibrated trust）的途径之一，但透明度与校准信任之间的关系尚不明确。Brooks 认为，为了实现人类与人工智能之间的有效合作，保持对人工智能的适当期望非常重要，但这不应以盲目乐观或不屑一顾的怀疑态度来处理 [55]。传达不确定性和局限性有助于人类建立准确的心理模型；然而，过度规避风险可能会在面对人工智能系统提供的宝贵辅助时，导致信心不足。有效的透明度需要在传达局限性时进行校准。

System failure and error recovery mechanisms can be identified as critical junctures for trust calibration. The actions taken by the AI system during such failure and its potential to help in the recovery from human errors can impact the overall level of trust. Alpay and Alpay examined the deficient human-in-the-loop oversight mechanisms in sophisticated AI systems and identified patterns that occur in such failure scenarios [59]. Their results show that trust violations resulting from unforeseen system failures are difficult to mitigate, especially when humans are not provided with clear explanations for the reasons behind these unforeseen system failures. Designing for graceful degradation can aid in ensuring trust levels are maintained in spite of unforeseen system failures in AI systems.系统故障和错误恢复机制可以被视为信任校准的关键节点。AI 系统在故障期间采取的行动及其在帮助从人为错误中恢复方面的潜力，可能会影响整体信任水平。Alpay 和 Alpay 审查了复杂 AI 系统中“人在回路”监督机制的不足，并识别了此类故障场景中出现的模式 [59]。他们的结果表明，由不可预见的系统故障导致的信任破坏难以缓解，特别是在人类没有获得这些故障背后原因的清晰解释时。针对“优雅降级”（graceful degradation）进行设计，有助于确保即使在 AI 系统发生不可预见的故障时，也能维持信任水平。

The organizational context in which human–AI collaboration occurs shapes trust dynamics in ways that extend beyond individual user–system interactions. James examined human-in-the-loop architectures for trustworthy AI planning in mission-critical business intelligence systems, emphasizing how organizational structures, accountability mechanisms, and cultural factors influence whether humans exercise meaningful oversight [60]. In an organizational setting, trust exists at various levels: individuals need to trust the AI system, individuals need to be trusted by the organization to provide adequate oversight, and the organization needs to trust that the overall human–AI system meets performance and safety requirements. Disalignment at these different levels can create oversight issues even if all levels appear to be functioning correctly in isolation.人机协作发生的组织环境以超越个体用户与系统交互的方式塑造了信任动态。James 审查了任务关键型商业智能系统中用于可信 AI 规划的“人在回路”架构，强调了组织结构、问责机制和文化因素如何影响人类是否行使了有效的监督 [60]。在组织环境中，信任存在于多个层面：个体需要信任 AI 系统，个体需要被组织信任以提供充分的监督，而组织需要信任整体的人机系统能够满足性能和安全要求。如果这些不同层面的目标不一致，即使所有层面在孤立状态下看起来运行正常，也可能产生监督问题。

Responsibility attribution in human–AI systems creates complex dynamics that affect trust and oversight behavior. When errors occur in collaborative human–AI processes, questions arise about whether responsibility lies with the AI system, the human operator, the system designers, or the organization that deployed the system [61]. Mellamphy discusses how different understandings of the relationship between humans and artificial intelligence, including humanistic and posthumanist understandings, imply different understandings of responsibility. The unclear sense of responsibility can lead to unwise human intervention due to a sense of unaccountability for artificial intelligence errors or can lead to obstructive behavior due to a sense of blame for uncontrollable artificial intelligence errors.人机系统中责任归属的问题产生了影响信任和监督行为的复杂动态。当人机协作流程中出现错误时，问题便随之产生：责任究竟在于 AI 系统、人类操作员、系统设计者，还是部署该系统的组织 [61]？Mellamphy 讨论了对人与人工智能之间关系的不同理解（包括人文主义和后人类主义的理解）如何暗示了对责任的不同理解。由于对人工智能错误缺乏问责感，这种不明确的责任感可能导致不明智的人为干预，或者由于对不可控的人工智能错误感到自责，导致阻碍性行为。

The scapegoat-in-the-loop concept captures situations where humans are nominally included in AI systems primarily to absorb responsibility rather than to provide meaningful oversight [61]. In these configurations, human involvement may satisfy legal or regulatory requirements without actually improving system safety or performance. Ottun and Flores conducted a review of human oversight and human-in-the-loop approaches, identifying characteristics that distinguish meaningful oversight from superficial compliance [2]. Meaningful oversight requires that humans have sufficient information, time, expertise, and authority to evaluate and override AI decisions, conditions that are not always met in practice despite nominal human-in-the-loop designs.“回路中的替罪羊”（scapegoat-in-the-loop）概念描述了这样一种情况：人类被名义上纳入 AI 系统，主要是为了承担责任，而非提供有效的监督 [61]。在这种配置下，人类的参与可能满足了法律或监管要求，但实际上并未提高系统的安全或性能。Ottun 和 Flores 对人类监督和“人在回路”方法进行了综述，识别出了区分有效监督与表面合规的特征 [2]。有效的监督要求人类拥有足够的信息、时间、专业知识和权限来评估并推翻 AI 的决策，尽管存在名义上的“人在回路”设计，但实践中并不总是能满足这些条件。

Adaptive methods for trust calibration attempt to control the behavior of a system by responding to patterns of trust from humans. In other words, rather than presenting the results of the AI systems in a uniform manner, it is possible for the presentation of the results or recommendations to be adjusted based on the trustworthiness of the results or the patterns of trust from individual users. This was demonstrated by Cho et al. in a wearable sensor for thermal comfort control [62]. Such adaptive approaches can help correct both over-trust and under-trust by providing stronger endorsements when AI confidence is high and more hedged recommendations when uncertainty is elevated.用于信任校准的自适应方法试图通过响应人类的信任模式来控制系统行为。换句话说，与其以统一的方式呈现 AI 系统的结果，不如根据结果的可信度或个体用户的信任模式，对结果或建议的呈现方式进行调整。Cho 等人在用于热舒适度控制的可穿戴传感器中证明了这一点 [62]。此类自适应方法可以通过在 AI 置信度高时提供更强力的支持，并在不确定性升高时提供更谨慎的建议，从而帮助纠正过度信任和信任不足。

The long-term sustainability of human-in-the-loop oversight requires attention to skill maintenance and engagement. When AI systems perform well consistently, human operators may experience skill decay in the tasks that AI has assumed, reducing their ability to detect errors or take over when AI systems fail [57]. Concurrently, a decrease in the cadence of substantive intervention opportunities could also foster boredom, which would further impair the quality of oversight. The development of human-in-the-loop systems for sustainable operation requires the intentional preservation of human skills and engagement, which could be done through training exercises, task allocations, or system designs that sustain substantive human involvement even when AI automation could be used for autonomous performance of tasks.“人在回路”监督的长期可持续性需要关注技能维护和参与度。当 AI 系统持续表现良好时，人类操作员可能会在 AI 已接管的任务上出现技能退化，从而降低他们在 AI 系统故障时检测错误或接管任务的能力 [57]。同时，实质性干预机会频率的降低也可能引发厌倦感，这会进一步损害监督质量。开发可持续运行的“人在回路”系统，需要有意识地保持人类的技能和参与度，这可以通过培训演练、任务分配或系统设计来实现，即使在 AI 自动化可用于自主执行任务时，也能维持人类的实质性参与。

4.1. Human-in-the-Loop Fairness4.1. “人在回路”的公平性

The concept of fairness in AI systems defies a technical description since different stakeholders have different and sometimes conflicting ideas about what is fair. Research by Nakao et al. was instrumental in laying the groundwork for involving end users in interactive human-in-the-loop AI fairness and showing that end users can provide valuable information for fairness criteria that experts might overlook [35]. The study also demonstrates that the assessment of fairness can be improved by considering diverse viewpoints and that interactive systems can play an important role in enabling the participation of non-technical stakeholders in decisions related to fairness. The results also reveal significant differences between user groups regarding their perceptions of fairness, which suggests that fairness metrics may not be sufficient to address the fairness concerns that are most important to users.AI 系统中公平性的概念难以用技术描述来定义，因为不同的利益相关者对什么是公平有着不同、有时甚至是相互冲突的看法。Nakao 等人的研究在让最终用户参与交互式“人在回路”AI 公平性方面奠定了基础，并表明最终用户可以为专家可能忽略的公平性标准提供有价值的信息 [35]。该研究还表明，通过考虑不同的观点可以改善公平性评估，交互式系统在促使非技术利益相关者参与公平性相关决策方面可以发挥重要作用。结果还揭示了不同用户群体在公平性感知方面存在显著差异，这表明公平性指标可能不足以解决用户最关心的公平性问题。

Participatory approaches to AI fairness seek to incorporate stakeholder perspectives throughout the system development lifecycle rather than treating fairness as a post-hoc evaluation criterion. Taka et al. developed methods for integrating stakeholder feedback to incorporate fairness perspectives in responsible AI development [63]. This approach recognizes that fairness concerns are rooted in social contexts and cannot be fully specified by system developers in a vacuum. By providing formal channels through which stakeholders can contribute, participatory fairness approaches aim to uncover concerns that would not be visible to system developers and ensure that the legitimacy of these systems is established among these social groups.AI 公平性的参与式方法旨在将利益相关者的观点融入整个系统开发生命周期，而不是将公平性视为事后的评估标准。Taka 等人开发了整合利益相关者反馈的方法，以便在负责任的 AI 开发中纳入公平性视角 [63]。这种方法认识到公平性问题植根于社会背景，无法由系统开发者在真空中完全界定。通过提供利益相关者可以贡献意见的正式渠道，参与式公平性方法旨在发现系统开发者无法察觉的问题，并确保这些系统在这些社会群体中建立合法性。

Crowdsourcing offers one mechanism for gathering diverse perspectives on AI fairness at scale. Nakao examined how crowdworkers’ characteristics and the framing of fairness metrics affect perceptions of AI fairness, finding that demographic factors, personal experiences, and the specific way fairness questions are posed all influence judgments [35]. This study also highlights the complexity involved in aggregating fairness judgments from diverse populations and the need for careful consideration in the process. Though the use of crowdsourced evaluations for fairness can be useful in augmenting the results obtained from experts by understanding the population’s perspective on the system’s behavior, it is important that the crowdsourced samples be representative.众包提供了一种大规模收集 AI 公平性多样化观点的机制。Nakao 研究了众包工作者的特征和公平性指标的框架如何影响对 AI 公平性的感知，发现人口统计学因素、个人经验以及提出公平性问题的具体方式都会影响判断 [35]。这项研究还强调了汇总来自不同人群的公平性判断所涉及的复杂性，以及在这一过程中进行仔细考量的必要性。尽管利用众包评估来了解人群对系统行为的看法，对于增强专家获取的结果很有用，但众包样本必须具有代表性这一点至关重要。

The transition from reactive fairness auditing to proactive fairness-aware design represents an important evolution in human-in-the-loop fairness research. Griffen and Owens proposed moving from traditional human-in-the-loop models to participatory systems of governance for AI in healthcare, arguing that meaningful fairness requires ongoing stakeholder engagement rather than one-time consultation [64]. The model that they have created conceptualizes the affected communities as collaborators in AI governance rather than merely being subjected to AI systems. There is a need for greater human engagement in AI governance than in the usual oversight mechanisms. There is a possibility that the values of the community can be aligned more closely with AI systems.从反应式公平性审计向主动式公平感知设计转型，是“人在回路”公平性研究中的一个重要演变。Griffen 和 Owens 建议从传统的“人在回路”模型转向医疗保健领域 AI 的参与式治理系统，认为有意义的公平性需要持续的利益相关者参与，而非一次性的咨询 [64]。他们创建的模型将受影响的社区视为 AI 治理的合作者，而非仅仅是 AI 系统的受体。在 AI 治理中，人类的参与度需要比常规监督机制更高。社区的价值观与 AI 系统之间存在更紧密对齐的可能性。

4.2. Bias Detection and Mitigation4.2. 偏见检测与缓解

Algorithmic bias may result from various factors like biased data, incorrect assumptions of the model, and evaluation metrics that do not correctly measure the differential impact. Human-in-the-loop strategies for algorithmic bias detection employ human judgment to detect biased outcomes that may be missed by automated metrics. The strategies also help to determine the difference between unfairness and disparities. Harris proved that the integration of human-in-the-loop systems with artificial intelligence fairness toolkits may help mitigate age bias in employment hiring algorithmic systems [38]. The research found that human involvement improved both the detection of bias and the development of effective remediation strategies.算法偏见可能由多种因素导致，如偏见数据、模型的不正确假设以及无法正确衡量差异化影响的评估指标。用于算法偏见检测的“人在回路”策略利用人类判断来检测自动化指标可能遗漏的偏见结果。这些策略还有助于确定不公平与差异之间的区别。Harris 证明，将“人在回路”系统与人工智能公平性工具包相结合，有助于缓解就业招聘算法系统中的年龄偏见 [38]。研究发现，人类的参与既改善了偏见检测，也促进了有效补救策略的制定。

Employment and hiring represent high-stakes domains where algorithmic bias can have severe consequences for individuals and where human oversight is particularly important. Neupane investigated algorithmic justice in AI and machine learning-enabled talent acquisition systems, examining how human-in-the-loop approaches can identify and address discrimination in automated hiring [65]. The study shows how the hiring algorithms can reflect biases related to age, gender, educational backgrounds, and other protected characteristics. These biases are often not immediately apparent by analyzing the results of the algorithms. With proper training, humans can recognize disparate treatment patterns that may not be immediately apparent through statistical analysis.就业和招聘是高风险领域，算法偏见会在此类领域对个人产生严重后果，因此人类监督尤为重要。Neupane 对 AI 和机器学习赋能的人才获取系统中的算法公正性进行了调查，研究了“人在回路”方法如何识别并解决自动化招聘中的歧视问题 [65]。研究显示，招聘算法如何反映与年龄、性别、教育背景及其他受保护特征相关的偏见。通过分析算法结果，这些偏见通常无法立即显现。经过适当培训，人类可以识别出统计分析可能无法立即显现的差异化对待模式。

Algamaty developed approaches for fair and transparent AI in hiring that combine resume–job matching with bias mitigation and human-in-the-loop auditing [66]. The framework also incorporates several checkpoints where human evaluators rate system recommendations for possible bias before decisions are made on candidates. The hierarchical system recognizes that bias can be introduced during various stages of the hiring process, and effective mitigation of this issue requires consideration of each of these stages. The human auditing mechanism is intended not only to pinpoint specific instances of biased recommendations but also to create feedback that can improve system performance over time.Algamaty 开发了用于招聘中公平且透明 AI 的方法，将简历与职位匹配、偏见缓解以及“人在回路”审计相结合 [66]。该框架还纳入了多个检查点，人类评估者在对候选人做出决定之前，会对系统建议进行潜在偏见评分。该分层系统认识到偏见可能在招聘过程的各个阶段被引入，而有效缓解此问题需要对这些阶段中的每一个进行考量。人类审计机制的目的不仅在于指出偏见建议的具体实例，还在于创建反馈，从而随时间推移改善系统性能。

Content moderation and online platform governance present distinctive challenges for bias detection where the boundaries between harmful content and protected speech are contested and context-dependent. Sheombar examined fallacies in online hate speech detection, revealing how both AI systems and human moderators can exhibit biases in identifying fringe hate speech [67]. The study shows that human-in-the-loop configurations can both reinforce or reduce the effects of bias in AI systems depending on the way human review is structured. For effective mitigation of bias in content moderation systems, it is essential that the biases present in human evaluators be taken into account along with the methods used for measuring alignment between human and AI bias so that unfair outcomes are not allowed to occur.内容审核和在线平台治理在偏见检测方面提出了独特的挑战，因为有害内容与受保护言论之间的界限存在争议且取决于具体语境。Sheombar 研究了在线仇恨言论检测中的谬误，揭示了 AI 系统和人类审核员在识别边缘仇恨言论时都可能表现出偏见 [67]。研究表明，根据人类审查的组织方式，“人在回路”配置既可以强化也可以减少 AI 系统中的偏见影响。为了有效缓解内容审核系统中的偏见，必须将人类评估者中存在的偏见，以及用于衡量人类与 AI 偏见之间对齐程度的方法纳入考量，以避免出现不公平的结果。

The issue of mitigating biases does not only apply to individual decisions but also to the system itself, which is a result of multiple decisions made using AI-assisted decisions. Joseph & Yakubu studied human-in-the-loop decisions for various decision-sensitive domains like education and the non-profit sector. They found that while biases are small individually, their aggregated effect can be substantial [68]. Their study also emphasizes the need to monitor aggregate outcomes as well as decisions made by individuals, and it suggests that feedback mechanisms that point to patterns of disparate impact need to be created. Relying solely on human oversight to review decisions made on a case-by-case basis may not be sufficient to point to biases that only become apparent when outcomes are reviewed on a wide range of decisions.缓解偏见的问题不仅适用于个体决策，也适用于系统本身，因为系统是多次使用 AI 辅助决策的结果。Joseph 和 Yakubu 研究了教育和非营利组织等多个决策敏感领域的“人在回路”决策。他们发现，虽然偏见在个体层面很小，但其累积效应可能相当大 [68]。他们的研究还强调了监测汇总结果以及个体决策的必要性，并建议需要创建能够指出差异化影响模式的反馈机制。仅依赖人类监督来逐案审查决策，可能不足以指出只有在对广泛决策的结果进行审查时才会显现的偏见。

The above studies collectively suggest that human oversight is not an inherently bias-reducing mechanism; rather, it is contingent upon design or governance decisions. Oversight is seen to be beneficial to bias mitigation to the extent that humans are provided with clear decision support criteria, fairness diagnostics, and sufficient time for review. Additionally, review bodies are better positioned to mitigate biases when they are diverse rather than representative of a single institution’s perspective. In such cases, humans are capable of detecting harms that are not captured by aggregate metrics of models and are able to take corrective actions [38,66].上述研究共同表明，人类监督并非一种天生就能减少偏见的机制；相反，它取决于设计或治理决策。监督在人类被提供清晰的决策支持标准、公平性诊断和充足审查时间的情况下，被认为有利于缓解偏见。此外，当审查机构具有多样性而非仅代表单一机构的观点时，它们更有能力缓解偏见。在这种情况下，人类有能力检测到模型汇总指标无法捕捉到的伤害，并能够采取纠正措施 [38,66]。

On the other hand, oversight can be unproductive when human reviewers face high-throughput pressure, vague guidance, or unclear definitions of policies. In such cases, reviewers can fall back on automation biases, replicate existing social biases, or normalize existing biases as “reasonable” outcomes. Such unproductive outcomes have been observed in Sheombar’s study on hate speech moderation, where both human and model biases can “reinforce” each other [67]. A coherent HITL bias-mitigation strategy therefore requires dual-level evaluation: case-level review quality and outcome-level disparity monitoring over time, with escalation triggers when the two diverge [68].另一方面，当人类审查员面临高吞吐量压力、模糊的指导或不明确的政策定义时，监督可能毫无成效。在这种情况下，审查员可能会退回到自动化偏见，复制现有的社会偏见，或将现有偏见常态化为“合理”的结果。Sheombar 在关于仇恨言论审核的研究中观察到了此类非生产性结果，其中人类偏见和模型偏见可以“相互强化” [67]。因此，一个连贯的 HITL 偏见缓解策略需要双层评估：案例层面的审查质量和随时间推移的产出层面差异监测，并在两者出现分歧时设置升级触发机制 [68]。

4.3. Value Alignment and Accountability4.3. 价值对齐与问责制

The challenge of aligning artificial intelligence (AI) systems with human values is not only related to fairness but also extends to other issues related to the goals that AI systems should pursue, as well as conflicts that may arise between these values. Chen et al. developed methodologies for creating ethical AI systems based on human-in-the-loop approaches, arguing that value alignment requires ongoing human involvement, not just initial goal specification [69]. Their model recognizes that human values are complex, situation-dependent, and sometimes internally contradictory, requiring means of human judgment to control system behavior when algorithmic methods are insufficient. Chen also examined the practicality and rationality of human-in-the-loop methods for AI value alignment, including situations where human-in-the-loop methods improve value alignment and situations where they create additional complexities [70].使人工智能（AI）系统与人类价值观对齐的挑战，不仅涉及公平性，还扩展到 AI 系统应追求的目标以及这些价值观之间可能产生的冲突等其他问题。Chen 等人开发了基于“人在回路”方法创建伦理 AI 系统的方案，认为价值对齐需要持续的人类参与，而不仅仅是初始的目标设定 [69]。他们的模型认识到人类价值观是复杂的、视情境而定的，有时甚至是内部矛盾的，因此当算法方法不足时，需要通过人类判断来控制系统行为。Chen 还研究了用于 AI 价值对齐的“人在回路”方法的实用性和合理性，包括“人在回路”方法改善价值对齐的情况，以及它们产生额外复杂性的情况 [70]。

Regulatory frameworks increasingly mandate human oversight for AI systems that affect fundamental rights and safety. Middleton et al. examined trust, regulation, and human-in-the-loop AI within the European region, analyzing how regulatory requirements shape the design and deployment of AI systems [71]. The study revealed the tension between the need for human oversight and the limits of human attention and expertise. The study by Constantino focused on the accountability issues that arise from the implementation of Article 14 in the EU AI Act, which calls for human oversight in high-risk AI systems in public administration [72]. The analysis revealed ambiguities in how oversight requirements should be implemented and questions about whether mandated human involvement actually improves outcomes or merely shifts liability.监管框架日益要求对影响基本权利和安全的 AI 系统进行人类监督。Middleton 等人研究了欧洲地区的信任、监管和“人在回路”AI，分析了监管要求如何塑造 AI 系统的设计和部署 [71]。研究揭示了人类监督的需求与人类注意力和专业知识局限性之间的紧张关系。Constantino 的研究聚焦于因实施欧盟《人工智能法案》第 14 条而产生的问责问题，该条款要求对公共管理中的高风险 AI 系统进行人类监督 [72]。分析揭示了在如何实施监督要求方面存在模糊性，并对强制性的人类参与究竟是改善了结果还是仅仅转移了责任提出了质疑。

The attribution of responsibility when AI-assisted decisions cause harm remains contested and has significant implications for accountability and governance. Ranisch examined the phenomenon of scapegoat-in-the-loop configurations in medical AI, where human involvement may serve primarily to absorb responsibility rather than to improve decision quality [73]. In that paper, concerns are raised about human-in-the-loop systems that satisfy formal accountability requirements yet do not provide actual oversight. In a study on the irreducibility of consciousness in human intelligence and its implications for accountability in artificial intelligence, Samarawickrama discussed how some aspects of moral responsibility cannot be handed over to a system, no matter how advanced it is [74]. These philosophical considerations inform debates about the appropriate scope of AI autonomy and the conditions under which human oversight is genuinely necessary.当 AI 辅助决策造成损害时，责任归属问题仍存在争议，并对问责制和治理具有重大影响。Ranisch 研究了医疗 AI 中“回路中的替罪羊”配置现象，在这种配置中，人类的参与可能主要用于承担责任，而非提高决策质量 [73]。在该论文中，人们对那些满足了形式上的问责要求却未提供实际监督的“人在回路”系统表示担忧。在关于人类智能中意识的不可还原性及其对人工智能问责制影响的研究中，Samarawickrama 讨论了某些道德责任方面是如何无法移交给系统的，无论系统多么先进 [74]。这些哲学思考为有关 AI 自主权适当范围以及人类监督在何种条件下真正必要的辩论提供了参考。

Ethical frameworks from diverse philosophical traditions offer resources for thinking about human–AI relationships and value alignment. Liu examined human-in-the-loop ethical AI for care robots through the lens of Confucian virtue ethics, demonstrating how non-Western ethical traditions can inform the design of AI systems that support human flourishing [75]. Tang proposed the Chiron Imperative as a framework for creating wise and just AI–human centaurs, drawing on classical concepts of practical wisdom to guide human–AI collaboration [15]. These cross-cultural and historically informed approaches expand the conceptual resources available for addressing value alignment and highlight the importance of diverse perspectives in shaping AI ethics.来自不同哲学传统的伦理框架为思考人机关系和价值对齐提供了资源。Liu 通过儒家德性伦理的视角研究了护理机器人的“人在回路”伦理 AI，展示了非西方伦理传统如何为支持人类繁荣的 AI 系统设计提供参考 [75]。Tang 提出了“喀戎准则”（Chiron Imperative）作为创建智慧且公正的“人机半人马”的框架，借鉴了实践智慧的经典概念来指导人机协作 [15]。这些跨文化和具有历史渊源的方法扩展了解决价值对齐问题的概念资源，并强调了多样化视角在塑造 AI 伦理方面的重要性。

Organizational and sectoral contexts shape how value alignment and accountability are operationalized in practice. Singh developed a governance framework for generative AI in banking that operationalizes trust through structured human-in-the-loop oversight [48]. The framework addresses the particular regulatory requirements and risk considerations of financial services while providing a model that could be adapted to other sectors. Joshi and Vaidya examined responsible AI adoption in small and medium enterprises, proposing frameworks that account for the resource constraints that smaller organizations face in implementing meaningful human oversight [76].组织和行业背景塑造了价值对齐和问责制在实践中如何运作。Singh 为银行业生成式 AI 开发了一个治理框架，通过结构化的“人在回路”监督将信任付诸实践 [48]。该框架在解决金融服务特定监管要求和风险考量的同时，提供了一个可适用于其他行业的模型。Joshi 和 Vaidya 研究了中小企业中负责任的 AI 采用情况，提出了考虑到较小组织在实施有效人类监督时面临资源限制的框架 [76]。

Privacy considerations interact with human-in-the-loop design in complex ways that affect both the feasibility and the ethics of human oversight. Rivadeneira et al. developed a unified privacy-preserving model for human-in-the-loop cyber–physical systems, addressing the challenge of maintaining human oversight while protecting sensitive data [77]. Their approach demonstrates that privacy and oversight need not be in fundamental tension but require careful technical design to achieve simultaneously. Anthuvan et al. examined human–AI collaboration in academic writing, developing the Scholarly HI-AI Loop Framework for ethical knowledge production that addresses questions of attribution, integrity, and accountability in AI-assisted research [8]. As AI systems become more capable of generating content that could be mistaken for human work, questions of transparency and appropriate disclosure become increasingly important for maintaining trust in knowledge production institutions. Table 6 summarizes key application domains of human-in-the-loop (HITL) AI, highlighting typical loop configurations, risk levels, and domain-specific challenges.隐私考量以复杂的方式与“人在回路”设计相互作用，影响了人类监督的可行性和伦理。Rivadeneira 等人开发了一个用于“人在回路”信息物理系统的统一隐私保护模型，解决了在保护敏感数据的同时维持人类监督的挑战 [77]。他们的方法表明，隐私和监督之间不必存在根本性的紧张关系，但需要精心的技术设计才能同时实现。Anthuvan 等人研究了学术写作中的人机协作，开发了用于伦理知识生产的“学术 HI-AI 回路框架”，解决了 AI 辅助研究中有关归属、完整性和问责制的问题 [8]。随着 AI 系统生成可能被误认为是人类作品的内容的能力不断增强，透明度和适当披露的问题对于维持知识生产机构的信任变得越来越重要。表 6 总结了“人在回路”（HITL）AI 的关键应用领域，突出了典型的回路配置、风险水平和特定领域的挑战。

5.1. Healthcare and Life Sciences5.1. 医疗保健与生命科学

The medical applications of AI in healthcare support a human-in-the-loop approach since decision-making in healthcare has a direct impact on health and the accountability requirements in healthcare mandate that only qualified individuals be responsible for healthcare. Bakken highlights the need for human involvement in health AI by arguing that the complexity of clinical reasoning and decision-making, as well as the relevance of the context in which patients receive healthcare and the ethical requirements in healthcare decision-making, requires involvement in AI [78]. This perspective reflects broader consensus in medical informatics that AI should augment rather than replace clinical judgment, with systems designed to support rather than supplant the expertise of healthcare professionals.AI 在医疗保健中的医学应用支持“人在回路”方法，因为医疗决策直接影响健康，且医疗问责制要求只有合格的个人才能对医疗保健负责。Bakken 强调了健康 AI 中人类参与的必要性，认为临床推理和决策的复杂性，以及患者接受医疗保健的环境的相关性，加上医疗决策中的伦理要求，使得 AI 必须有人的参与 [78]。这一观点反映了医学信息学界的广泛共识，即 AI 应增强而非取代临床判断，系统设计应旨在支持而非取代医疗专业人员的专业知识。

Medical imaging represents one of the most active areas for HITL AI development in healthcare. Yu et al. developed PI-RADSAI, a human-in-the-loop model for prostate cancer diagnosis based on MRI that integrates radiologist expertise with machine learning capabilities [26]. The system presents AI-generated assessments to radiologists who can confirm, modify, or reject the automated analysis based on their clinical judgment and additional patient information not available to the algorithm. Wu et al. demonstrated AI-accelerated structuring of radiology reports with human oversight, showing how AI can reduce documentation burden while maintaining the accuracy and completeness that clinical communication requires [88]. These applications illustrate the pattern of AI handling routine processing while humans focus on interpretation, verification, and communication.医学影像代表了医疗保健中 HITL AI 开发最活跃的领域之一。Yu 等人开发了 PI-RADSAI，这是一种基于 MRI 的前列腺癌诊断“人在回路”模型，它将放射科医生的专业知识与机器学习能力相结合 [26]。该系统向放射科医生展示 AI 生成的评估结果，医生可以根据临床判断以及算法无法获取的其他患者信息，确认、修改或拒绝自动化分析。Wu 等人展示了在人类监督下 AI 加速结构化放射科报告的过程，显示了 AI 如何在减轻文档负担的同时，保持临床沟通所需的准确性和完整性 [88]。这些应用说明了 AI 处理常规流程，而人类专注于解释、验证和沟通的模式。

Neurological applications have demonstrated the potential for HITL systems to achieve performance that generalizes across diverse clinical settings. Yang et al. developed a human-in-the-loop AI system for clinical seizure recognition that achieved continental generalization, maintaining diagnostic accuracy across patient populations in different healthcare systems [79]. The human-in-the-loop part was also essential in managing cases in which the automated detection was still in doubt. The study showed that human involvement can help address the generalization issues that automated detection systems face.神经学应用展示了 HITL 系统实现跨不同临床环境的性能泛化潜力。Yang 等人开发了一种用于临床癫痫识别的“人在回路”AI 系统，实现了跨大陆的泛化，在不同医疗系统的患者群体中保持了诊断准确性 [79]。在自动化检测仍存在疑问的情况下，“人在回路”部分也至关重要。研究表明，人类的参与可以帮助解决自动化检测系统面临的泛化问题。

Clinical decision support systems represent another important application area where human-in-the-loop design principles inform system architecture. Steffny et al. developed a human-in-the-loop centered AI-based clinical decision support system for professional care planning, emphasizing the importance of designing AI assistance that aligns with clinical workflows and decision-making processes [80]. Theilmann et al. examined success factors for AI in healthcare, identifying human-in-the-loop integration as a key determinant of whether AI systems achieve their intended benefits in clinical practice [89]. These studies highlight that technical performance alone does not guarantee clinical value and that effective integration with human practitioners requires attention to workflow, interface design, and organizational factors.临床决策支持系统是“人在回路”设计原则指导系统架构的另一个重要应用领域。Steffny 等人开发了一种以“人在回路”为中心的 AI 辅助临床决策支持系统，用于专业护理规划，强调了设计与临床工作流程和决策过程相一致的 AI 辅助的重要性 [80]。Theilmann 等人审查了医疗保健中 AI 的成功因素，将“人在回路”整合确定为 AI 系统能否在临床实践中实现预期收益的关键决定因素 [89]。这些研究强调，仅靠技术性能并不能保证临床价值，与人类从业者的有效整合需要关注工作流程、界面设计和组织因素。

Notably, the acceptance of human-in-the-loop (HITL) systems by medical professionals can differ substantially from one medical domain to another, as can the type of task. This difference is often more related to the workflow compatibility than the actual model accuracy. In many cases, system deployment failure is related to the presentation of AI results at inappropriate points within the care pathway, increasing documentation burdens, or the uncompensated verification demands placed on the clinician under time pressures. This can result in the system, under these circumstances, reverting to a state of ’rubber-stamping’ or bypassing the system, which can lead to a failure of safety and acceptance. Conversely, the acceptance of the system can be facilitated by the integration with existing decision points, the provision of explanations at a clinically relevant level of granularity, and the establishment of clear escalation and accountability processes [80,89,90]. This evidence suggests that workflow integration is not a secondary implementation detail but a primary determinant of whether healthcare HITL AI delivers real-world benefit.值得注意的是，医疗专业人员对“人在回路”（HITL）系统的接受度在不同医学领域之间可能存在显著差异，任务类型也是如此。这种差异通常更多地与工作流程的兼容性有关，而非模型本身的准确性。在许多情况下，系统部署失败与 AI 结果在护理路径中不恰当节点的呈现、文档负担的增加，或在时间压力下给临床医生带来的未经补偿的验证需求有关。在这种情况下，系统可能会退化为“橡皮图章”式的形式主义或被绕过，从而导致安全性和接受度的失败。相反，通过与现有决策点的整合、提供临床相关粒度水平的解释，以及建立清晰的升级和问责流程，可以促进对系统的接受 [80,89,90]。这些证据表明，工作流程整合并非次要的实施细节，而是医疗 HITL AI 能否带来现实世界效益的主要决定因素。

Healthcare applications in resource-constrained settings present particular challenges and opportunities for HITL AI. Kabata and Thaldar examined human-in-the-loop requirements for AI healthcare applications in low-resource settings, where the scarcity of medical expertise makes AI assistance potentially more valuable but also raises concerns about appropriate oversight [3]. This suggests that Human-in-the-Loop (HITL) design for low-resource contexts should take into consideration the lack of access to expert human evaluators and possibly involve alternative forms of oversight. Fahad and Huang suggested a framework for continuous validation of generative AI in healthcare. The need for this arose from the limitation of sustaining vigilant human oversight in the face of generally reliable AI outputs [32].资源受限环境下的医疗保健应用为 HITL AI 带来了特殊的挑战和机遇。Kabata 和 Thaldar 研究了低资源环境下 AI 医疗应用的“人在回路”要求，在这些环境下，医疗专业知识的匮乏使得 AI 辅助具有潜在的更高价值，但也引发了对适当监督的担忧 [3]。这表明，针对低资源环境的“人在回路”（HITL）设计应考虑缺乏专家人类评估者的情况，并可能涉及其他形式的监督。Fahad 和 Huang 提出了一个用于医疗保健中生成式 AI 持续验证的框架。这一需求源于在面对通常可靠的 AI 输出时，难以维持警惕的人类监督的局限性 [32]。

The organizational dimensions of human-in-the-loop healthcare AI extend beyond individual clinical encounters to encompass institutional governance and quality assurance. Herrmann and Pfeiffer argued for keeping the organization in the loop as a general concept for human-centered AI, using medical imaging as an illustrative example [90]. Their framework recognizes that effective human oversight requires not only capable individual practitioners but also organizational structures that support monitoring, feedback, and continuous improvement. Griffen and Owens proposed moving from traditional human-in-the-loop models to participatory systems of governance for AI in healthcare, envisioning patient and community involvement in shaping how AI systems are developed and deployed [64].“人在回路”医疗 AI 的组织维度超出了个体临床接触的范畴，涵盖了机构治理和质量保证。Herrmann 和 Pfeiffer 主张将组织保持在回路中作为以人为本的 AI 的通用概念，并以医学影像为例进行了说明 [90]。他们的框架认识到，有效的监督不仅需要有能力的个体从业者，还需要支持监测、反馈和持续改进的组织结构。Griffen 和 Owens 建议从传统的“人在回路”模型转向医疗保健领域 AI 的参与式治理系统，设想患者和社区参与塑造 AI 系统的开发和部署方式 [64]。

Emerging applications in pathology and laboratory medicine demonstrate the expanding scope of HITL healthcare AI. Guo et al. evaluated cell AI foundation models in kidney pathology using human-in-the-loop enrichment, developing methods for pathologists to guide model improvement through targeted feedback on challenging cases [91]. Lin et al. applied human-in-the-loop AI screening for hepatic porphyria diagnosis, demonstrating potential improvements over standard diagnostic approaches [92]. Kandala et al. developed cross-lingual mental health ontologies for Indian languages using explainable AI and human-in-the-loop validation, addressing the challenge of extending AI capabilities to underserved linguistic communities [93]. These applications illustrate continuing expansion of HITL approaches into new clinical domains.病理学和实验室医学中的新兴应用展示了 HITL 医疗 AI 不断扩大的范围。Guo 等人利用“人在回路”富集技术评估了肾脏病理学中的细胞 AI 基础模型，开发了病理学家通过对具有挑战性的案例进行有针对性的反馈来指导模型改进的方法 [91]。Lin 等人应用“人在回路”AI 筛查肝卟啉症诊断，展示了优于标准诊断方法的潜在改进 [92]。Kandala 等人利用可解释 AI 和“人在回路”验证，为印度语言开发了跨语言心理健康本体，解决了将 AI 能力扩展到服务不足的语言社区的挑战 [93]。这些应用说明了 HITL 方法在新的临床领域中的持续扩展。

5.2. Autonomous Systems and Robotics5.2. 自主系统与机器人

One such domain where the balance between the capability of artificial intelligence and human oversight is a safety concern is that of autonomous vehicles. In such a case, human-in-the-loop design should be considered on a spectrum of SAE levels of automation, as opposed to a binary concept. Lower levels of automation are typically those where humans are in a constant loop of control assistance, while higher levels of automation leave humans in a role of monitoring with occasional fallback intervention. As levels of automation increase, the bottleneck in human performance shifts from vehicle control proficiency to attention, situational awareness, and preparedness for intervention under time pressure [39].人工智能能力与人类监督之间的平衡属于安全问题的领域之一是自动驾驶汽车。在这种情况下，“人在回路”设计应被视为 SAE 自动化分级频谱上的一个概念，而非二元概念。较低级别的自动化通常是人类处于持续的控制辅助回路中，而较高级别的自动化则让人类处于监控角色，并偶尔进行备用干预。随着自动化水平的提高，人类表现的瓶颈从车辆控制熟练度转移到了注意力、态势感知以及在时间压力下进行干预的准备程度 [39]。

A key conceptual distinction is between training-time and deployment-time HITL paradigms. Training-time approaches, such as human-guided reinforcement learning and mentor-style correction, use human input to shape policy learning before deployment and to reduce unsafe exploration during development [28,29]. Paradigms related to deployment time, on the other hand, relate to supervisory control, takeover, and response in real-time operational traffic. These paradigms are not interchangeable since they differ in terms of required cognitive resources, failure modes, and levels of regulatory interest. The equivalency of these paradigms obscures critical trade-offs inherent in safety assurance.一个关键的概念区别在于训练阶段和部署阶段的 HITL 范式。训练阶段的方法，如人类引导的强化学习和导师式修正，利用人类输入在部署前塑造策略学习，并在开发过程中减少不安全的探索 [28,29]。另一方面，与部署阶段相关的范式涉及实时交通运营中的监督控制、接管和响应。这些范式不可互换，因为它们在所需认知资源、故障模式和监管关注水平方面存在差异。这些范式的等同性掩盖了安全保证中固有的关键权衡。

Recent empirical work on occupant intervention behavior under extreme driving conditions further underscores this distinction. Xu et al. show that intervention decisions depend on perceived risk trajectory, cue timing, and human confidence in automation status, not only on objective hazard intensity [94]. Related driver-in-the-loop evidence indicates that collaboration quality changes over time as users adapt, which affects both trust calibration and takeover performance [95]. For HITL evaluation, this implies that autonomous-driving systems should be assessed with paradigm-specific metrics (e.g., intervention timing, missed versus unnecessary interventions, recovery quality after takeover) rather than only aggregate task success.近期关于极端驾驶条件下乘员干预行为的实证研究进一步强调了这一区别。Xu 等人表明，干预决策取决于感知的风险轨迹、提示时机和人类对自动化状态的信心，而不仅仅取决于客观的危险强度 [94]。相关的驾驶员在回路证据表明，协作质量会随着用户适应而随时间变化，这会影响信任校准和接管表现 [95]。对于 HITL 评估而言，这意味着自动驾驶系统应使用范式特定的指标（如干预时机、漏掉与不必要的干预、接管后的恢复质量）进行评估，而不是仅评估汇总的任务成功率。

Self-driving laboratories represent an emerging application domain that combines autonomous experimentation with human scientific judgment. Hysmith et al. examined the future of self-driving laboratories, exploring the progression from human-in-the-loop interactive AI to gamification approaches that can engage broader communities in guiding automated scientific discovery [96]. These systems largely automate the experimental procedures while still relying on human scientists to generate hypotheses, interpret results, and control the direction of the research. The human-in-the-loop component ensures that automated experiments align with scientific goals and that unexpected results are properly addressed.自动驾驶实验室是一个新兴的应用领域，它将自主实验与人类科学判断相结合。Hysmith 等人审视了自动驾驶实验室的未来，探讨了从“人在回路”交互式 AI 到可以吸引更广泛社区引导自动化科学发现的博弈化方法的演进 [96]。这些系统在很大程度上自动化了实验程序，同时仍然依赖人类科学家来生成假设、解释结果并控制研究方向。“人在回路”组件确保了自动化实验与科学目标保持一致，并确保意外结果得到妥善处理。

Drone and unmanned aerial vehicle applications have motivated substantial HITL research due to their operational complexity and potential for harm. Lopes conducted studies on operator fatigue, trust, and workload demand in human-in-the-loop AI-enabled drone systems, revealing how extended operation affects human oversight quality [57]. The study also shows that fatigue affects not only trust calibration but also performance, which is a clear indication of the need for effective workload management when it comes to sustaining effective human performance during operations. Inoguchi et al. proposed various workflows for roof damage detection using drones in a collaborative framework between humans and AI [97].无人机和无人驾驶飞行器应用由于其操作复杂性和潜在危害，激发了大量的 HITL 研究。Lopes 对“人在回路”AI 赋能无人机系统中的操作员疲劳、信任度和工作负载需求进行了研究，揭示了长时间操作如何影响人类监督质量 [57]。研究还表明，疲劳不仅影响信任校准，还影响表现，这清楚地表明在维持操作期间有效的人类表现时，需要进行有效的工作负载管理。Inoguchi 等人提出了在人机协作框架下，使用无人机进行屋顶损坏检测的各种工作流程 [97]。

Robotics applications in care and service contexts raise distinctive questions about the appropriate relationship between humans and AI systems. Liu examined human-in-the-loop ethical AI for care robots, drawing on Confucian virtue ethics to develop frameworks for robots that support human flourishing rather than merely performing assigned tasks [75]. Ali explored how human-in-the-loop approaches can enhance safety and adaptability in interactive AI robotic systems, emphasizing the importance of mechanisms for humans to guide and correct robot behavior in dynamic environments [98]. These applications require HITL designs that support nuanced human–robot interaction rather than simple supervisory oversight.护理和服务环境中的机器人应用提出了有关人与 AI 系统之间适当关系的独特问题。Liu 研究了护理机器人的“人在回路”伦理 AI，借鉴儒家德性伦理来开发支持人类繁荣而非仅仅执行指定任务的机器人框架 [75]。Ali 探索了“人在回路”方法如何增强交互式 AI 机器人系统的安全性和适应性，强调了人类在动态环境中引导和纠正机器人行为的机制的重要性 [98]。这些应用需要支持细致入微的人机交互，而非简单的监督式“人在回路”设计。

The transportation infrastructure applications of human-in-the-loop (HITL) autonomous systems expand upon the single-vehicle setting by considering broader traffic management objectives. Previati et al. created simulation frameworks for roundabout traffic scenarios that incorporate automated vehicles, artificial intelligence, edge computing, and human-in-the-loop components to study the integration of human oversight in complex traffic scenarios that involve multiple autonomous agents [99]. Happer examined human-in-the-loop versus fully autonomous AI systems for crisis-driven defense electronics manufacturing, analyzing trade-offs between automation efficiency and human adaptability in high-pressure production environments [100].“人在回路”（HITL）自主系统的交通基础设施应用通过考虑更广泛的交通管理目标，扩展了单车设置。Previati 等人创建了包含自动驾驶车辆、人工智能、边缘计算和“人在回路”组件的环岛交通场景仿真框架，以研究在涉及多个自主代理的复杂交通场景中人类监督的整合 [99]。Happer 研究了用于危机驱动型国防电子制造业的“人在回路”与全自主 AI 系统，分析了在高压生产环境中自动化效率与人类适应性之间的权衡 [100]。

5.3. Cybersecurity and Critical Infrastructure5.3. 网络安全与关键基础设施

Cybersecurity is a domain wherein the adversarial nature of cyberthreats and the dynamic nature of cyberattacks present unique challenges for the functioning of AI systems without human intervention. Karunamurthy et al. examined human-in-the-loop intelligence for advancing AI-centric cybersecurity by arguing that cybersecurity requires the integration of AI-based pattern recognition with human expertise on attackers’ motivations and organizational context [81]. Their analysis emphasizes that cybersecurity threats often involve social engineering and exploitation of human factors that AI systems struggle to model, making human judgment essential for comprehensive threat assessment.网络安全是一个领域，其中网络威胁的对抗性本质和网络攻击的动态性，为人工智能系统在无人干预下的运行带来了独特的挑战。Karunamurthy 等人通过论证网络安全需要将基于人工智能的模式识别与人类对攻击者动机及组织背景的专业知识相结合，探讨了旨在推进以人工智能为中心的网络安全的“人在回路”（human-in-the-loop）智能 [81]。他们的分析强调，网络安全威胁往往涉及社会工程学和对人工智能系统难以建模的人为因素的利用，这使得人类判断对于全面的威胁评估至关重要。

The integration of human expertise into AI-driven cybersecurity operations requires careful attention to workflow design and decision support. Owen et al. developed approaches for proactive AI in cybersecurity with human-in-the-loop collaboration for intelligent threat detection and alerting [18]. Their approach utilizes artificial intelligence to prioritize potential threats and offer relevant context to human analysts. The human has the authority to decide the response actions. Turner et al. studied human-in-the-loop decision-making for AI-based cyber defense. The authors examined the interaction between security analysts and AI-based recommendations for cyberdefense. The authors also examined factors that impact human judgment on AI-based cyberdefense [83].将人类专业知识整合到人工智能驱动的网络安全运营中，需要仔细考量工作流设计和决策支持。Owen 等人开发了用于网络安全的主动式人工智能方法，通过“人在回路”协作实现智能威胁检测和报警 [18]。他们的方法利用人工智能对潜在威胁进行优先级排序，并向人类分析师提供相关背景信息。人类拥有决定应对行动的权力。Turner 等人研究了基于人工智能的网络防御中的“人在回路”决策。作者考察了安全分析师与基于人工智能的网络防御建议之间的交互，并分析了影响人类对基于人工智能的网络防御判断的因素 [83]。

Critical infrastructure protection presents high-stakes applications where the consequences of both successful attacks and false alarms can be severe. Campbell et al. developed human-in-the-loop adaptive AI cybersecurity frameworks for safety-critical infrastructure systems, addressing the challenge of maintaining security while avoiding disruptions caused by overly aggressive automated responses [82]. da Silva examined AI-driven cybersecurity with a human-in-the-loop approach, proposing methods for integrating human expertise into automated security operations centers [101]. These applications require HITL designs that support rapid human response while avoiding alert fatigue that could cause analysts to miss genuine threats.关键基础设施保护属于高风险应用，成功的攻击和误报所带来的后果都可能非常严重。Campbell 等人针对安全关键型基础设施系统开发了“人在回路”自适应人工智能网络安全框架，旨在解决在保持安全性的同时避免因过度激进的自动化响应而导致中断的挑战 [82]。da Silva 研究了采用“人在回路”方法的人工智能驱动的网络安全，提出了将人类专业知识整合到自动化安全运营中心的方法 [101]。这些应用需要能够支持人类快速响应，同时避免可能导致分析师错过真实威胁的“警报疲劳”的“人在回路”设计。

Software development security has emerged as an application area where generative AI capabilities create both opportunities and risks. Sharma et al. developed cybersecurity-aware human-in-the-loop test orchestration for AI-powered DevSecOps, examining how human oversight can be integrated into automated development pipelines to catch security vulnerabilities before deployment [102]. Konakanchi examined human-in-the-loop secure code synthesis, addressing the challenge of ensuring that AI-generated code does not introduce security vulnerabilities [103]. These applications recognize that AI code generation capabilities must be paired with appropriate security review to avoid introducing new attack surfaces.软件开发安全已成为一个应用领域，生成式人工智能的能力既带来了机遇，也带来了风险。Sharma 等人开发了具备网络安全意识的“人在回路”测试编排系统，用于人工智能驱动的开发运维（DevSecOps），并研究了如何将人工监督整合到自动化开发流水线中，以便在部署前发现安全漏洞 [102]。Konakanchi 研究了“人在回路”的安全代码合成，旨在解决如何确保人工智能生成的代码不会引入安全漏洞的挑战 [103]。这些应用认识到，人工智能的代码生成能力必须与适当的安全审查相结合，以避免引入新的攻击面。

5.4. Finance, Education, and Industry5.4. 金融、教育与工业

This subsection considers the logics of finance, education, and industry, which utilize the same HITL labels, though with different optimization objectives, error cost, and accountability structures. The logics are as follows: finance focuses on legal defensibility and rights protection, education focuses on pedagogical validity and equity, while industry focuses on throughput, quality consistency, and safety. The analytical point is that the effectiveness of the “human-in-the-loop” is not necessarily a guarantee, as its effectiveness depends on the alignment of the points of human intervention with the dominant risks.本小节探讨了金融、教育和工业的逻辑，它们虽然使用相同的“人在回路”（HITL）标签，但具有不同的优化目标、错误成本和问责结构。其逻辑如下：金融侧重于法律可辩护性和权利保护；教育侧重于教学有效性和公平性；而工业则侧重于吞吐量、质量一致性和安全性。分析的要点在于，“人在回路”的有效性并非必然得到保证，因为其有效性取决于人类干预点与主要风险点之间的一致性。

With regard to finance, especially in lending and associated decision processes, the primary concern is one of algorithmic accountability rather than the sheer number of human checks. Human override layers can contribute to greater fairness only in combination with decision rationales, explainability for adverse actions, and monitoring for disparate impact across protected groups. Without these components, human checks can be nothing more than a buffer for liabilities while patterns of biased decision processes persist. The governance-focused frameworks developed by Joshi and Singh can only be taken seriously as institutional controls such as documentation governance, escalation governance, and review governance [48,84]. This aligns with broader regulatory debates on meaningful human oversight and accountability design under high-risk AI governance, where policy compliance and substantive fairness can diverge [65,71,72].在金融领域，特别是在贷款及相关决策过程中，首要关注的是算法问责制，而非单纯的人工检查数量。人工覆盖层只有与决策依据、针对不利行动的可解释性以及对受保护群体差异化影响的监控相结合，才能有助于实现更大的公平性。如果没有这些组成部分，人工检查充其量只是责任的缓冲，而有偏见的决策过程模式依然会持续存在。Joshi 和 Singh 开发的以治理为中心的框架只有作为诸如文档治理、升级治理和审查治理等机构控制手段时，才能被认真对待 [48,84]。这与高风险人工智能治理下关于有意义的人工监督和问责制设计的更广泛的监管辩论相一致，在这些辩论中，政策合规性与实质性公平可能会产生分歧 [65,71,72]。

The main challenge in the academic setting is how to maintain the legitimacy of assessment while utilizing the potential of artificial intelligence for scalability. Human involvement in the process can help in better judgment; however, this can also lead to instructor inconsistency or institutional bias if the rubrics are not defined well. There is evidence that in all use cases involving grading and content generation, human-in-the-loop design should be aligned to rubrics and moderation for equitable outcomes for different student groups [9,85,86,87,104]. In this setting, human oversight is effective when it is structured as calibrated academic judgment rather than discretionary exception handling.学术环境中的主要挑战在于，如何在利用人工智能的扩展潜力时保持评估的合法性。过程中引入人为参与有助于做出更好的判断；然而，如果评分标准定义不明确，这也可能导致教师评分不一致或制度偏见。有证据表明，在所有涉及评分和内容生成的用例中，“人在回路”的设计应与评分标准和审核保持一致，以确保不同学生群体获得公平的结果 [9,85,86,87,104]。在这种环境下，当人工监督被构建为经过校准的学术判断，而非随意性的例外处理时，它是有效的。

In industrial or manufacturing settings, human-in-the-loop (HITL) systems are typically assessed for their reliability under production stress. The review process improves defect detection and adaptability to changing environments, but performance suffers when interfaces are overly burdensome cognitively or when operators are relegated to passive monitoring for exceptions. Empirical work on manufacturing and visual inspection settings suggests that successful implementation relies on strong interplay between explainability, training, and feedback mechanisms for continually refining predictive models and practices [23,41]. Across all three domains, the common lesson is that analytical review should focus on how oversight is operationalized—authority, timing, evidence, and feedback—rather than on whether a human is nominally present in the loop.在工业或制造环境中，“人在回路”（HITL）系统通常根据其在生产压力下的可靠性进行评估。审查过程改善了缺陷检测和对不断变化环境的适应能力，但当界面认知负担过重，或者操作员被降级为仅进行被动监控以处理异常情况时，性能会受到影响。关于制造和视觉检查环境的实证研究表明，成功的实施依赖于可解释性、培训和反馈机制之间的紧密互动，以不断优化预测模型和实践 [23,41]。在这三个领域中，共同的教训是，分析性审查应关注监督是如何运作的——即权力、时机、证据和反馈——而不是关注人类是否名义上处于回路中。

5.5. Cross-Scenario Common Challenges in High-Risk HITL Deployments5.5. 高风险 HITL 部署中跨场景的共同挑战

In all these domains—healthcare, autonomous systems, cybersecurity, finance, education, and industry—these recurrent problems follow a common pattern despite the varying objectives of each domain. Firstly, there is a problem in establishing accountability in a distributed manner among different models, operators, and organizations. There is a need for clear override privileges and decision logs to avoid diffusion of responsibility. Secondly, trust calibration is another problem that is easily disrupted by over-trust or under-trust. Over-trust leads to automation bias, whereas under-trust leads to a decrease in the effectiveness or usage of the systems. Thirdly, cognitive load and fatigue limit the actual effectiveness of operators in providing oversight.在所有这些领域——医疗保健、自动驾驶系统、网络安全、金融、教育和工业——尽管每个领域的目标各异，但这些反复出现的问题遵循着共同的模式。首先，在不同的模型、操作员和组织之间以分布式方式建立问责制存在问题。需要明确的覆盖权限和决策日志，以避免责任分散。其次，信任校准是另一个容易被过度信任或信任不足所破坏的问题。过度信任会导致自动化偏见，而信任不足则会导致系统有效性或使用率下降。第三，认知负荷和疲劳限制了操作员在提供监督方面的实际有效性。

Fourthly, the timing of human interventions is just as important as the interventions themselves. In other words, delayed or untimely interventions may be just as detrimental to safety despite the formal human-in-the-loop structures that are in place. Fifthly, feedback has a different quality that may include noise, bias, or even strategic behavior that may remain unresolved. This has a significant impact on the reliability of the models. Lastly, there are institutional constraints that affect the feasibility of different types of oversight. Overall, these cross-scenario challenges point to the fact that human-in-the-loop systems need to be viewed as part of an organizational control system rather than just a model–human interface.第四，人类干预的时机与干预本身同样重要。换句话说，尽管存在正式的“人在回路”结构，但延迟或不适时的干预对安全性造成的损害可能同样严重。第五，反馈具有不同的质量，可能包含噪声、偏见，甚至可能存在未得到解决的策略性行为。这对模型的可靠性有重大影响。最后，存在影响不同类型监督可行性的制度约束。总体而言，这些跨场景的挑战表明，“人在回路”系统需要被视为组织控制系统的一部分，而不仅仅是模型与人类的接口。

6.1. Interaction Design Principles6.1. 交互设计原则

The design of human–AI interaction includes considerations related to the presentation of information, the elicitation of human input, and the design and communication of the allocation of tasks between humans and artificial intelligence. Okuboye examined the redesign of global business processes to optimize the collaboration between artificial intelligence and employees, revealing principles for structuring tasks to take advantage of the strengths of both human and artificial intelligence [10]. The research found that successful HITL implementations require explicit attention to task allocation, with clear delineation of which decisions remain with humans and which are delegated to automated systems. Ambiguity in role boundaries was identified as a significant source of implementation failure.人机交互的设计包括与信息呈现、人类输入获取以及人类与人工智能之间任务分配的设计与沟通相关的考量。Okuboye 研究了全球业务流程的重构，以优化人工智能与员工之间的协作，揭示了旨在利用人类和人工智能各自优势的任务结构化原则 [10]。研究发现，成功的 HITL 实施需要明确关注任务分配，明确界定哪些决策由人类保留，哪些委托给自动化系统。角色边界的模糊性被确定为实施失败的重要根源。

Prompt design has emerged as a critical interaction design challenge for systems based on large language models. Ranade et al. demonstrated how rhetorical strategies can be systematically applied to design prompts that make AI more useful for human users [31]. Their method involves a human–AI communication concept that can take advantage of the principles of good rhetoric, which are already established. This can change the way prompt engineering is done from a trial-and-error approach to a more communicative approach, which can have implications for how users are taught to use the system.提示词设计已成为基于大语言模型的系统所面临的关键交互设计挑战。Ranade 等人展示了如何系统地应用修辞策略来设计提示词，使人工智能对人类用户更有用 [31]。他们的方法涉及一种人机沟通概念，可以利用已建立的良好修辞原则。这可以将提示词工程的方式从试错法转变为一种更具沟通性的方法，这可能会对用户如何学习使用系统产生影响。

Attention management represents a fundamental concern in HITL design, particularly for safety-critical applications where lapses in human attention can have severe consequences. Nicosia and Kristensson developed design principles for AI-assisted attention-aware systems in human-in-the-loop safety-critical applications [105]. Their approach tackles the problem of maintaining adequate human vigilance as the AI system performs well in routine cases. This is because it recognizes that the success of the AI system may contribute to the erosion of the human vigilance it originally required. The principles emphasize the design of sustained human engagement rather than simply providing information.注意力管理是 HITL 设计中的一个基本问题，特别是对于那些人类注意力疏忽可能导致严重后果的安全关键型应用而言。Nicosia 和 Kristensson 为“人在回路”安全关键型应用中人工智能辅助的注意力感知系统制定了设计原则 [105]。他们的方法解决了当人工智能系统在常规情况下表现良好时，如何保持人类足够警惕的问题。这是因为它认识到，人工智能系统的成功可能会导致其最初所需的人类警惕性的削弱。这些原则强调设计持续的人类参与，而不仅仅是提供信息。

Additional complementary empirical evidence from studies in autonomous vehicles can further clarify these requirements for interaction design. In a study by Xu et al., the authors simulated the riders’ interventions during high-level autonomous driving in extreme conditions and found that the probability of a rider’s intervention depends not only on the actual severity of the hazard but also on the trajectory of perceived risks and the timing of system cues and the occupants’ confidence in the state of the automation [94]. This line of inquiry is socially relevant given observed differences in occupant intervention behaviors, which suggest that some riders are systematically disadvantaged by one-size-fits-all takeover interface designs. In the context of human-in-the-loop (HITL) design, this implies that supervisory interfaces should be designed to incorporate early risk communication, graduated levels of intervention, and fallbacks, rather than relying on late-stage binary takeover requests.来自自动驾驶汽车研究的补充性实证证据可以进一步阐明这些交互设计要求。在 Xu 等人的一项研究中，作者模拟了乘客在极端条件下进行高级自动驾驶时的干预情况，发现乘客干预的概率不仅取决于危险的实际严重程度，还取决于感知风险的轨迹、系统提示的时机以及乘客对自动化状态的信心 [94]。鉴于观察到的乘客干预行为的差异，这一研究方向具有社会意义，这表明一些乘客因“一刀切”的接管界面设计而受到系统性不利影响。在“人在回路”（HITL）设计的背景下，这意味着监督界面应设计为包含早期风险沟通、分级干预和后备方案，而不是依赖后期的二元接管请求。

The temporal structure of human–AI interaction affects both the quality of human contributions and the efficiency of collaborative workflows. Ou et al. conducted a case study on revealing and explaining human–AI interaction loop failures, identifying patterns of breakdown that occur when interaction timing, feedback cycles, or task sequencing are poorly designed [106]. Their analysis suggests that one of the most troublesome types of failure modes identified by their model are infinite loops where humans and AI systems are constantly deferring to one another without ever resolving the issue. Effective human-in-the-loop (HITL) design needs to address such failure modes by carefully thinking through the interaction flow and providing explicit mechanisms to resolve deadlocks.人机交互的时间结构既影响人类贡献的质量，也影响协作工作流的效率。Ou 等人通过对揭示和解释人机交互回路故障的案例研究，确定了当交互时机、反馈周期或任务排序设计不佳时会发生的故障模式 [106]。他们的分析表明，其模型识别出的最麻烦的故障模式之一是无限循环，即人类和人工智能系统不断地相互推诿，而从未解决问题。有效的“人在回路”（HITL）设计需要通过仔细考虑交互流程并提供明确的死锁解决机制来应对此类故障模式。

Conversational interfaces present distinctive design challenges for HITL systems that rely on natural language interaction. Sun developed approaches for conversational interfaces cooperating with AI and monitoring technology using human-in-the-loop interaction for intelligent behavioral intervention [107]. The study examines the effects of dialogue design on user engagement, trust, and information exchanged between humans and artificial intelligence systems. The authors of this study, namely Tseng et al., developed AI chatbots for team-based diabetes care by applying these principles through an iterative human-in-the-loop process, showing how this is done in the specific case of healthcare [108].对话式界面为依赖自然语言交互的 HITL 系统带来了独特的设计挑战。Sun 开发了用于对话式界面的方法，这些界面与人工智能和监控技术协作，利用“人在回路”交互进行智能行为干预 [107]。该研究考察了对话设计对用户参与度、信任度以及人类与人工智能系统之间信息交换的影响。该研究的作者 Tseng 等人通过迭代的“人在回路”过程应用这些原则，开发了用于团队糖尿病护理的人工智能聊天机器人，展示了如何在医疗保健这一特定案例中实现这一点 [108]。

The comparison of different HITL approaches reveals trade-offs that inform design decisions. Yin et al. compared human-learning HITL approaches with machine-learning HITL approaches in co-design contexts, finding that each design philosophy has distinct strengths depending on task characteristics and user expertise [109]. The human learning approach, in which the artificial intelligence plays a part in guiding human learning or instruction, was found to be more effective for tasks that require skill development. On the other hand, machine learning approaches, in which humans provide training data for the artificial intelligence, were more appropriate for tasks that require adapting to user preferences. A good understanding of these factors can help in determining the appropriate human-in-the-loop configuration for a given application.不同 HITL 方法的比较揭示了影响设计决策的权衡。Yin 等人在共同设计背景下比较了人类学习型 HITL 方法与机器学习型 HITL 方法，发现每种设计哲学根据任务特征和用户专业知识都有其独特的优势 [109]。人类学习方法（其中人工智能在指导人类学习或教学中发挥作用）被发现对于需要技能发展的任务更有效。另一方面，机器学习方法（其中人类为人工智能提供训练数据）更适合需要适应用户偏好的任务。对这些因素的良好理解有助于确定给定应用所需的“人在回路”配置。

User experience design for HITL systems must balance efficiency with factors that sustain human engagement and capability over time. Ahi et al. developed a UX-centric human-in-the-loop system for product lifecycle management that achieved substantial improvements in reviewer productivity while maintaining decision quality [110]. Their approach suggests that productivity improvements should not be at the cost of human skill loss or disengagement. Rather, it is argued that interface design should complement human skills rather than substitute for them. The study proves that careful UX design can lead to not only efficiency benefits but also to sustainable human–AI collaboration.HITL 系统的用户体验设计必须在效率与维持人类长期参与和能力所需的因素之间取得平衡。Ahi 等人为产品生命周期管理开发了一个以用户体验为中心的“人在回路”系统，在保持决策质量的同时，显著提高了审查员的生产力 [110]。他们的方法表明，生产力的提高不应以牺牲人类技能的丧失或参与度的降低为代价。相反，有人认为界面设计应补充人类技能，而不是替代它们。该研究证明，仔细的用户体验设计不仅可以带来效率上的好处，还可以实现可持续的人机协作。

6.2. Human State Monitoring and Adaptive Interfaces6.2. 人类状态监控与自适应界面

The performance of humans in human-in-the-loop systems is dynamic, depending on their cognitive state, which includes factors such as fatigue, workload, attention, and emotional state. The ability of systems that can monitor and adapt to the state of humans allows for effective collaboration under dynamic states of performance. Lopes carried out extensive research on operator fatigue, trust, and workload demands in AI-based drone systems that are integrated into human-in-the-loop systems [57]. The study revealed a high level of performance degradation in fatigue conditions and identified physiological and behavioral measures that can be used by adaptive systems to detect performance degradation in human capabilities. The results have implications for any human-in-the-loop (HITL) scenario involving human prolonged operation.人类在“人在回路”系统中的表现是动态的，取决于其认知状态，其中包括疲劳、工作负荷、注意力和情绪状态等因素。能够监控并适应人类状态的系统，使得在动态的性能状态下进行有效协作成为可能。Lopes 对集成到“人在回路”系统中的人工智能无人机系统中的操作员疲劳、信任和工作负荷需求进行了广泛研究 [57]。研究揭示了在疲劳条件下性能会严重下降，并确定了自适应系统可用于检测人类能力性能下降的生理和行为指标。研究结果对任何涉及人类长期操作的“人在回路”（HITL）场景都有启示意义。

Risk management in attention-aware HITL systems requires mechanisms for detecting when human attention has degraded and for intervening appropriately. Nicosia and Kristensson extended their earlier design principles to address risk management in human-in-the-loop AI-assisted attention-aware systems [111]. The framework identifies the type of attention-related risk, the methodologies for monitoring each type of attention-related risk, and the strategies for intervention that range from the least intrusive actions to the most decisive actions. The response to attentional slips detected depends on the level of interest involved as well as the existence of alternative mechanisms.注意力感知型 HITL 系统中的风险管理需要相应的机制，以检测人类注意力何时下降并进行适当的干预。Nicosia 和 Kristensson 扩展了他们早期的设计原则，以解决“人在回路”人工智能辅助注意力感知系统中的风险管理问题 [111]。该框架确定了注意力相关风险的类型、监控每种注意力相关风险的方法论，以及从最不具侵入性的行动到最果断的行动的干预策略。对检测到的注意力疏忽的响应取决于所涉及的利益水平以及是否存在替代机制。

Adaptive interfaces that respond to detected human states offer the potential for HITL systems that maintain effectiveness across varying conditions. Chivapong examined the strength and challenges of hybrid intelligence approaches, including adaptive mechanisms that adjust AI behavior based on observed human performance [5]. The analysis revealed both technical challenges in effectively inferring human states and design challenges in identifying suitable adaptations. Adaptations that are overly aggressive may compromise human agency, while those that are overly subtle may not effectively address performance degradation.响应检测到的人类状态的自适应界面，为在不同条件下保持有效性的 HITL 系统提供了可能。Chivapong 研究了混合智能方法的优势和挑战，包括根据观察到的人类表现调整人工智能行为的自适应机制 [5]。分析揭示了在有效推断人类状态方面的技术挑战，以及在确定适当适应方式方面的设计挑战。过于激进的适应可能会损害人类的主体性，而过于微妙的适应则可能无法有效解决性能下降问题。

Workload management in HITL systems involves not only monitoring current workload but also designing task allocation strategies that maintain sustainable human engagement. Benedikt et al. conducted a case study of human-in-the-loop AI in government, examining how workload considerations affect the design and implementation of AI-assisted public services [112]. The study revealed that early deployments were found to underestimate human workload demands, leading to bottlenecks in system performance. Sustainable human-in-the-loop (HITL) deployment requires an appraisal of human capabilities and the establishment of mechanisms for managing human workload demands.HITL 系统中的工作负荷管理不仅涉及监控当前的工作负荷，还涉及设计能够维持可持续人类参与的任务分配策略。Benedikt 等人对政府部门的“人在回路”人工智能进行了案例研究，考察了工作负荷考量如何影响人工智能辅助公共服务的设计和实施 [112]。研究表明，早期的部署低估了人类的工作负荷需求，导致了系统性能瓶颈。可持续的“人在回路”（HITL）部署需要对人类能力进行评估，并建立管理人类工作负荷需求的机制。

The design of systems for low-literacy or resource-constrained settings presents particular challenges for human state monitoring and adaptive interfaces. Adewale examined human-in-the-loop AI for community health workers, co-designing decision support systems for low-literacy settings [113]. The research emphasized that monitoring and adaptation mechanisms must be appropriate to user capabilities and context, avoiding assumptions about technological familiarity or interface conventions that may not hold across diverse user populations. Inclusive HITL design requires attention to the full range of potential users rather than optimization for typical or expert users alone.为低识字率或资源受限环境设计系统，在人类状态监控和自适应界面方面提出了特殊的挑战。Adewale 研究了社区卫生工作者的“人在回路”人工智能，为低识字率环境共同设计了决策支持系统 [113]。研究强调，监控和适应机制必须适合用户的能力和背景，避免对可能不适用于不同用户群体的技术熟悉度或界面惯例做出假设。包容性的 HITL 设计需要关注所有潜在用户，而不是仅仅针对典型用户或专家用户进行优化。

6.3. Evaluation Methods and Metrics6.3. 评估方法与指标

The evaluation of HITL systems requires a coherent framework that treats performance as a joint property of humans, AI models, interfaces, and organizational context rather than as model accuracy alone. Building on prior domain studies, we organize evaluation around five dimensions: (i) task effectiveness, (ii) human factors, (iii) interaction process quality, (iv) safety/fairness/governance outcomes, and (v) lifecycle robustness [68,114,115].HITL 系统的评估需要一个连贯的框架，将性能视为人类、人工智能模型、界面和组织背景的共同属性，而不仅仅是模型准确性。基于先前的领域研究，我们将评估组织为五个维度：(i) 任务有效性，(ii) 人为因素，(iii) 交互过程质量，(iv) 安全性/公平性/治理结果，以及 (v) 生命周期稳健性 [68,114,115]。

For task effectiveness, metrics should include domain performance (e.g., error rate, calibration, quality scores) together with productivity terms such as time-to-decision and throughput. For human factors, evaluation should capture workload, trust calibration, acceptance, and skill retention, because high nominal accuracy can coexist with degraded human vigilance or deskilling. For interaction process quality, useful indicators include intervention timing, override frequency, disagreement resolution quality, and rates of unnecessary versus missed interventions, especially in safety-critical settings [94,95]. For safety, fairness, and governance, systems should be assessed on subgroup disparities, auditability, explanation adequacy, and accountability traceability in real workflows [112,116]. For lifecycle robustness, longitudinal monitoring should test drift sensitivity, adaptation effects, and stability of human–AI collaboration after extended use [46].对于任务有效性，指标应包括领域性能（例如错误率、校准、质量得分）以及生产力指标，如决策时间和吞吐量。对于人为因素，评估应涵盖工作负荷、信任校准、接受度和技能保持，因为高名义准确性可能与人类警惕性下降或技能退化并存。对于交互过程质量，有用的指标包括干预时机、覆盖频率、分歧解决质量以及不必要干预与错过干预的比率，特别是在安全关键型设置中 [94,95]。对于安全性、公平性和治理，系统应在真实工作流中评估子群体差异、可审计性、解释充分性以及问责可追溯性 [112,116]。对于生命周期稳健性，纵向监控应测试漂移敏感性、适应效果以及长期使用后人机协作的稳定性 [46]。

Methodologically, this framework implies multi-stage evaluation rather than single-shot benchmarking: controlled experiments for mechanism identification, field studies for workflow realism, and longitudinal audits for sustainability. Experimental comparisons remain essential for isolating effects of interface and explanation design [116,117], while domain-specific studies clarify which metric trade-offs are acceptable in practice [80,118]. Importantly, reporting should make trade-offs explicit: gains in speed or throughput should be interpreted jointly with changes in oversight quality, fairness, and human cognitive burden.从方法论上讲，该框架意味着多阶段评估而非单次基准测试：用于识别机制的受控实验、用于工作流真实性的实地研究，以及用于可持续性的纵向审计。实验性比较对于隔离界面和解释设计的影响仍然至关重要 [116,117]，而领域特定研究则阐明了在实践中哪些指标权衡是可以接受的 [80,118]。重要的是，报告应明确指出权衡：速度或吞吐量的提升应与监督质量、公平性和人类认知负担的变化共同解读。

A practical reporting template is to report results as a vector of dimension-wise results rather than an aggregate score. This helps to avoid the suppression of failure modes where one dimension is improving while another is getting worse (e.g., improved decision speed but reduced calibration of human trust). The use of a structured evaluation profile helps to compare results across domains while preserving contextual differences and supports governance decisions related to the readiness of a human-in-the-loop (HITL) system for deployment, restricted deployment, or redesign.一种实用的报告模板是将结果报告为维度结果的向量，而不是汇总分数。这有助于避免掩盖故障模式，即一个维度在改善而另一个维度在恶化（例如，决策速度提高但人类信任校准降低）。使用结构化的评估概况有助于在保持上下文差异的同时比较跨领域的结果，并支持与“人在回路”（HITL）系统部署、受限部署或重新设计准备情况相关的治理决策。

Longitudinal and stress-condition evaluation is therefore essential for revealing adaptation effects, calibration drift, and context-sensitive breakdowns that cross-sectional evaluations can miss. Table 7 summarizes the open challenges and future research directions in human-in-the-loop (HITL) AI across technical, cognitive, organizational, and societal dimensions.因此，纵向和压力条件下的评估对于揭示横截面评估可能遗漏的适应效果、校准漂移和上下文敏感型故障至关重要。表 7 总结了“人在回路”（HITL）人工智能在技术、认知、组织和社会维度上的待解决挑战和未来研究方向。

7.1. Human-in-the-Loop in AI Regulation7.1. 人工智能监管中的“人在回路”

The most comprehensive set of regulations on the human oversight of artificial intelligence systems is provided by the European Union’s AI Act, which outlines different sets of responsibilities depending on the level of risk. According to Article 14 of the EU’s AI Act, human oversight is necessary for high-risk artificial intelligence systems; thus, these systems must be designed in a way that facilitates supervision by natural persons during operation. In her study on the accountability issues that are linked to the imperatives of Article 14 on the supervision of high-risk artificial intelligence systems in the public sector, Constantino found that there are considerable ambiguities [72]. The analysis identified tensions between regulatory demands for meaningful human control and practical constraints on human attention, expertise, and decision-making capacity that affect whether mandated oversight actually improves outcomes.欧盟的《人工智能法案》提供了关于人工智能系统人工监督的最全面监管集，该法案根据风险级别概述了不同的责任集。根据欧盟《人工智能法案》第 14 条，高风险人工智能系统需要人工监督；因此，这些系统的设计必须便于自然人在运行期间进行监督。Constantino 在研究公共部门高风险人工智能系统监督第 14 条要求所带来的问责问题时发现，存在相当大的模糊性 [72]。分析确定了监管对有意义的人类控制的需求，与人类注意力、专业知识和决策能力等实际约束之间的紧张关系，这些约束影响着强制性的监督是否确实改善了结果。

A broader regulatory comparison is necessary to provide context for these European Union developments. In contrast, in the United States, policy instruments such as Executive Order 14110 and the National Institute of Standards and Technology’s AI Risk Management Framework highlight governance processes, risk documentation, and monitoring, but not a single cross-sector legal architecture. These approaches operationalize oversight through risk functions that govern, map, measure, manage, and lifecycle implementation guidance for agencies, which can inform human-in-the-loop (HITL) development even if language related to “human-in-the-loop” is not as specific as that of Article 14 [119,120]. Stringent requirements are being clearly defined in sector-specific regulatory regimes. In the healthcare sector, for example, the U.S. Food and Drug Administration (FDA) provides detailed recommendations for software as a medical device with respect to AI/ML technologies, which stress the importance of comprehensive controls throughout the entire lifecycle, post-market surveillance, and change management plans. In the context of autonomous vehicles, recommendations from the U.S. Department of Transportation (DOT), National Highway Traffic Safety Administration (NHTSA), stress aspects like development of safety cases, human factors, fallback, and transparent safety assessments [121,122]. Together, these frameworks suggest that regulatory robustness for HITL systems depends on cross-framework alignment between legal accountability, risk-management process maturity, and domain-specific safety governance.需要进行更广泛的监管比较，为这些欧盟的发展提供背景。相比之下，在美国，第 14110 号行政命令和国家标准与技术研究院（NIST）的人工智能风险管理框架等政策工具强调治理流程、风险文档和监控，但没有单一的跨部门法律架构。这些方法通过治理、映射、衡量、管理和生命周期实施指导的风险职能来运作监督，即使相关“人在回路”的措辞不像第 14 条那样具体，这些方法也可以为“人在回路”（HITL）开发提供参考 [119,120]。严格的要求正在特定行业的监管制度中得到明确定义。例如，在医疗保健领域，美国食品药品监督管理局（FDA）针对人工智能/机器学习技术作为医疗器械的软件提供了详细建议，强调了整个生命周期中全面控制、上市后监督和变更管理计划的重要性。在自动驾驶汽车背景下，美国交通部（DOT）和国家公路交通安全管理局（NHTSA）的建议强调了安全案例开发、人为因素、后备方案和透明安全评估等方面 [121,122]。总之，这些框架表明，HITL 系统的监管稳健性取决于法律问责制、风险管理流程成熟度以及特定领域安全治理之间的跨框架一致性。

The implementation of regulatory oversight requirements in specific sectors reveals the complexity of translating general principles into operational practice. Lundberg et al. examined human-in-the-loop AI requirements for future unified air traffic management systems, analyzing how aviation safety regulations shape the design of AI-assisted air traffic control [123]. The study proves that the present safety culture and regulatory regime in the aviation industry provide avenues for integrating artificial intelligence while ensuring human accountability. Nevertheless, the development of new artificial intelligence capabilities requires a modification of traditional regulatory practices. Carvell et al. propose a human-in-the-loop testing framework for artificial intelligence agents used in air traffic control systems by using regulated testing methods that explain the means for testing if the artificial intelligence systems meet the regulations for human oversight [124].在特定行业实施监管监督要求，揭示了将一般原则转化为操作实践的复杂性。Lundberg 等人研究了未来统一空中交通管理系统的“人在回路”人工智能要求，分析了航空安全法规如何塑造人工智能辅助空中交通管制的设计 [123]。研究证明，航空业现有的安全文化和监管制度为整合人工智能同时确保人类问责制提供了途径。然而，新人工智能能力的发展需要对传统监管实践进行修改。Carvell 等人通过使用受监管的测试方法，为空中交通管制系统中使用的智能体提出了“人在回路”测试框架，这些方法解释了测试人工智能系统是否符合人工监督法规的手段 [124]。

Regulatory frameworks must address not only the presence of human oversight but also its effectiveness in achieving intended safety and accountability goals. Fahad and Huang developed a framework for the continuous validation of the results of generative AI systems, which are used for healthcare purposes [32].监管框架不仅必须解决人工监督的存在问题，还必须解决其在实现预期的安全和问责目标方面的有效性。Fahad 和 Huang 为医疗保健用途的生成式人工智能系统的结果持续验证开发了一个框架 [32]。

The extension of regulatory attention to generative AI systems has created new challenges for human oversight frameworks developed with traditional AI applications in mind. Singh proposed a governance framework for generative AI in banking that operationalizes trust through structured human-in-the-loop oversight at multiple stages of content generation and deployment [48]. The framework focuses on the unique issues that are present when controlling systems that are not fully predictable in their outcomes and whose risk is heavily dependent on the context of use. Anthuvan et al. proposed the Scholarly HI-AI Loop Framework for ethical AI-based knowledge production, which deals with governance issues that are unique to a scholarly environment where issues of integrity and attribution are of primary concern [8].监管关注点向生成式人工智能系统的扩展，为考虑到传统人工智能应用而开发的人工监督框架带来了新的挑战。Singh 为银行业生成式人工智能提出了一个治理框架，通过在内容生成和部署的多个阶段进行结构化的“人在回路”监督来将信任付诸实践 [48]。该框架侧重于控制结果不完全可预测且风险在很大程度上取决于使用背景的系统时所出现的独特问题。Anthuvan 等人提出了用于基于人工智能的道德知识生产的学术 HI-AI 回路框架，该框架处理了学术环境中特有的治理问题，其中诚信和归属问题是首要关注点 [8]。

7.2. Organizational Governance7.2. 组织治理

For human oversight to be successful, both competent practitioners and an organizational system that supports monitoring, accountability, and improvement are necessary. The strategy of engaging organizations rather than focusing only on individuals acknowledges that successful oversight is based on organizational capacity and culture, not just the vigilance of individual operators. Chivapong examined the strengths and weaknesses of hybrid intelligence approaches from an organizational perspective, including the factors that affect an organization’s ability to support successful human oversight as artificial intelligence improves [5].为了使人工监督取得成功，既需要称职的从业者，也需要支持监控、问责和改进的组织系统。采取与组织接触而非仅关注个人的策略，承认了成功的监督基于组织能力和文化，而不仅仅是单个操作员的警惕性。Chivapong 从组织角度研究了混合智能方法的优势和劣势，包括影响组织在人工智能改进时支持成功人工监督能力的因素 [5]。

The design of governance frameworks for specific organizational contexts requires attention to sector-specific requirements, existing processes, and stakeholder expectations. Joshi and Vaidya developed a framework for empowering responsible AI adoption in small and medium enterprises, recognizing that resource constraints shape what governance mechanisms are feasible for smaller organizations [76]. The focus is on proportionate governance that provides oversight without creating any burden that may impede the adoption of beneficial AI systems. The concept of AI-DAPT was proposed by Koussouris et al. for data and AIOps integration for future AI development, incorporating human-in-the-loop integration for governance considerations for AI operations in technology environments [125].针对特定组织背景的治理框架设计，需要关注行业特定要求、现有流程和利益相关者的期望。Joshi 和 Vaidya 为赋能中小企业负责任的人工智能采用开发了一个框架，认识到资源限制决定了哪些治理机制对较小组织是可行的 [76]。重点在于比例治理，即在不产生任何可能阻碍有益人工智能系统采用的负担的情况下提供监督。Koussouris 等人提出了 AI-DAPT 概念，用于未来人工智能开发的数据和 AIOps 集成，并将“人在回路”集成纳入了技术环境中人工智能运营的治理考量 [125]。

Governance frameworks must anticipate and address potential failures in human oversight mechanisms. Sarrat and Gomez examined human-in-the-loop vulnerabilities arising from social engineering at the intersection of AI and critical infrastructure [126]. The analysis showed that the human components in human-in-the-loop systems could be vulnerable to adversarial manipulation, where attackers exploit trust relationships and cognitive biases to compromise the effectiveness of the human oversight. The analysis of the vulnerability shows that there is a need for security considerations for the human oversight mechanisms, in addition to those for the AI systems.治理框架必须预见并解决人工监督机制中潜在的失败。Sarrat 和 Gomez 研究了人工智能与关键基础设施交叉点上因社会工程学而产生的“人在回路”漏洞 [126]。分析表明，“人在回路”系统中的人类组件可能容易受到对抗性操纵，攻击者利用信任关系和认知偏见来损害人工监督的有效性。对该漏洞的分析表明，除了对人工智能系统进行安全考量外，还需要对人工监督机制进行安全考量。

The scalability of human oversight presents governance challenges as AI systems are deployed across larger numbers of decisions and contexts. Ustalov examined challenges in data production for AI with human-in-the-loop involvement, identifying bottlenecks that emerge when human annotation or validation cannot keep pace with AI system requirements [127]. Huang et al. proposed efficient human-in-the-loop active learning as a framework for data labeling that addresses scalability through strategic allocation of human effort [21]. These technical approaches to scalability have governance implications because they affect the feasibility of maintaining meaningful human oversight as AI deployment expands.随着人工智能系统部署在越来越多的决策和背景中，人工监督的可扩展性带来了治理挑战。Ustalov 研究了涉及“人在回路”的人工智能数据生产挑战，确定了当人类标注或验证无法跟上人工智能系统需求时出现的瓶颈 [127]。Huang 等人提出高效的“人在回路”主动学习作为一种数据标注框架，通过战略性分配人类工作量来解决可扩展性问题 [21]。这些技术性的可扩展性方法具有治理意义，因为它们影响着随着人工智能部署扩展而维持有意义的人工监督的可行性。

7.3. Future Governance Directions7.3. 未来治理方向

The evolution of AI capabilities and deployment contexts continues to challenge existing governance frameworks and to motivate development of new approaches. Hysmith et al. examined the future of self-driving laboratories, exploring governance implications of the progression from human-in-the-loop interactive AI to gamification approaches that engage broader communities [96]. Their analysis suggests that future governance may need to accommodate diverse forms of human involvement beyond traditional expert oversight, potentially including citizen participation in guiding AI development and deployment decisions.人工智能能力和部署背景的演变继续挑战现有的治理框架，并推动了新方法的开发。Hysmith 等人研究了自动驾驶实验室的未来，探讨了从“人在回路”交互式人工智能转向吸引更广泛社区的博弈化方法所带来的治理影响 [96]。他们的分析表明，未来的治理可能需要容纳传统专家监督之外的多种人类参与形式，可能包括公民参与指导人工智能的开发和部署决策。

Emerging application domains present governance questions that existing frameworks may not adequately address. Mavrakis et al. examined integration of human-in-the-loop AI to tackle space communication delay challenges, where the physical constraints of space operations require governance approaches adapted to extreme latency and limited communication bandwidth [128]. Rodrigues et al. proposed digital health-enabled community-centered care as a scalable model utilizing human-in-the-loop AI for community health workers, raising governance questions about oversight in resource-constrained healthcare settings [129]. de Miranda explored AI companions for philosophical health using a human-in-the-loop framework, addressing governance considerations for AI applications in personal wellbeing and meaning-making contexts [130].新兴应用领域提出了现有框架可能无法充分解决的治理问题。Mavrakis 等人研究了整合“人在回路”人工智能以应对空间通信延迟挑战，其中空间操作的物理约束要求治理方法适应极端延迟和有限的通信带宽 [128]。Rodrigues 等人提出了数字健康赋能的社区中心化护理，作为一种利用“人在回路”人工智能为社区卫生工作者服务的可扩展模型，提出了资源受限医疗环境中监督的治理问题 [129]。de Miranda 使用“人在回路”框架探索了用于哲学健康的人工智能伴侣，解决了人工智能在个人福祉和意义构建背景下应用的治理考量 [130]。

The development of unified frameworks for HITL governance represents an ongoing research and policy challenge. Tang proposed the Chiron Imperative as a framework of six human-in-the-loop-based systems for creating wise and just AI–human centaurs, offering conceptual resources for thinking about governance across diverse application contexts [15]. Kovalerchuk conducted a critical review of interpretable AI for high-stakes tasks with human-in-the-loop, identifying future trends that will shape governance requirements [4]. Tsiakas and Murray-Rust used human-in-the-loop and explainable AI to envisage new future work practices, examining how governance frameworks must evolve to address changing relationships between humans and AI in workplace contexts [20].HITL 治理统一框架的开发是一项持续的研究和政策挑战。Tang 提出了“喀戎命令”（Chiron Imperative），作为一种基于“人在回路”的六系统框架，用于创建智慧和公正的人工智能-人类半人马，为思考不同应用背景下的治理提供了概念资源 [15]。Kovalerchuk 对具有“人在回路”的高风险任务的可解释人工智能进行了批判性审查，确定了将塑造治理要求的未来趋势 [4]。Tsiakas 和 Murray-Rust 利用“人在回路”和可解释人工智能设想了新的未来工作实践，研究了治理框架必须如何演变以应对工作场所背景下人类与人工智能之间不断变化的关系 [20]。

The political economy of the human-in-the-loop requirement can have a bearing on the development of governance. Regulatory requirements to implement a human-in-the-loop mandate may incur a cost of compliance, which may not be favorable to smaller players or those operating in a resource-scarce environment. On the contrary, it may provide a protective cloak to larger players who can afford the cost of governance. Atreyapurapu proposed a human-in-the-loop artificial intelligence framework for scalable online brand protection [131]. Marculescu and Silva examined emerging edge AI for human-in-the-loop cyber-physical systems, highlighting how distributed computing architectures create new governance challenges that centralized frameworks may not address [132]. Atkinson proposed nested human-in-the-loop AI using chain of code prompting for research tool development, illustrating how governance frameworks must accommodate increasingly sophisticated patterns of human–AI interaction [33].“人在回路”要求的政治经济学可能对治理的发展产生影响。实施“人在回路”授权的监管要求可能会产生合规成本，这对较小的参与者或在资源匮乏环境中运营的参与者可能不利。相反，它可能为能够承担治理成本的大型参与者提供保护伞。Atreyapurapu 提出了用于可扩展在线品牌保护的“人在回路”人工智能框架 [131]。Marculescu 和 Silva 研究了用于“人在回路”信息物理系统的新兴边缘人工智能，强调了分布式计算架构如何带来集中式框架可能无法解决的新治理挑战 [132]。Atkinson 提出了使用代码链提示进行研究工具开发的嵌套式“人在回路”人工智能，说明了治理框架必须如何适应日益复杂的人机交互模式 [33]。

Campbell et al. developed human-in-the-loop adaptive AI cybersecurity frameworks for safety-critical infrastructure systems, addressing governance requirements that arise when AI systems protect essential services [82]. Oye et al. proposed frameworks for mitigating diagnostic errors in AI-driven radiology through human-in-the-loop approaches, demonstrating how governance frameworks must address error management and quality assurance [133]. Assadi and Safaei examined interpretable AI in human–machine systems through the lens of product recommendation engines, revealing governance considerations that arise when AI systems influence consumer decisions at scale [53]. These diverse applications illustrate the breadth of contexts where HITL governance frameworks must operate and the challenge of developing approaches that can accommodate such diversity while maintaining meaningful oversight standards.Campbell 等人针对安全关键型基础设施系统开发了“人在回路”自适应人工智能网络安全框架，解决了人工智能系统保护基本服务时产生的治理要求 [82]。Oye 等人提出了通过“人在回路”方法减轻人工智能驱动的放射学诊断错误框架，展示了治理框架必须如何解决错误管理和质量保证问题 [133]。Assadi 和 Safaei 通过产品推荐引擎的视角研究了人机系统中的可解释人工智能，揭示了人工智能系统大规模影响消费者决策时出现的治理考量 [53]。这些多样化的应用说明了 HITL 治理框架必须运行的广阔背景，以及开发既能适应这种多样性同时又能维持有意义的监督标准的方案所面临的挑战。

8.1. Layered Future Research Agenda Aligned with the HITL Taxonomy8.1. 与 HITL 分类法一致的分层未来研究议程

To ensure a more systematic approach for future work, four interrelated layers can be followed and aligned with the aforementioned taxonomy’s dimensions: loop placement and interaction granularity. The technical layer’s priority areas include uncertainty estimation, learning in the presence of disagreement, secure feedback mechanisms, and adaptive escalation strategies. The aforementioned aspects directly affect the loop placement and interaction granularity. The cognitive layer’s priority areas include trust calibration, human workload-aware interface design, and mitigation of human bias and deskilling. The aforementioned aspects affect the interaction granularity. The organizational layer’s priority areas include governance capabilities, staffing models, accountability and traceability, audit processes, and escalation processes. The aforementioned aspects affect the human-in-the-loop design’s viability. The ethical and institutional layer’s priority areas include fairness in heterogeneous feedback mechanisms, transparency requirements, value pluralism management, and sector-specific regulations. The aforementioned aspects affect the human-in-the-loop design’s viability. The aforementioned aspects suggest that future human-in-the-loop research should focus on evaluating the performance of human-in-the-loop methods in terms of not only their predictive performance but also their viability regarding human control calibration at different risk levels and interaction densities.为了确保未来工作有更系统的方法，可以遵循四个相互关联的层级，并与上述分类法的维度（回路位置和交互粒度）保持一致：技术层的优先领域包括不确定性估计、存在分歧时的学习、安全反馈机制和自适应升级策略。上述方面直接影响回路位置和交互粒度。认知层的优先领域包括信任校准、人类工作负荷感知界面设计以及减轻人类偏见和技能退化。上述方面影响交互粒度。组织层的优先领域包括治理能力、人员配置模型、问责制和可追溯性、审计流程和升级流程。上述方面影响“人在回路”设计的可行性。伦理和制度层的优先领域包括异构反馈机制中的公平性、透明度要求、价值多元化管理和行业特定法规。上述方面影响“人在回路”设计的可行性。上述方面表明，未来的“人在回路”研究应侧重于评估“人在回路”方法的性能，不仅要评估其预测性能，还要评估其在不同风险水平和交互密度下人类控制校准的可行性。

8.2. Scalability of Human Oversight8.2. 人工监督的可扩展性

The scalability of human oversight is one of the key issues in the effective implementation of human-in-the-loop (HITL) technology. The more the AI technology is extended to different decision scenarios and contexts, the more the human oversight capability is limited. The current solutions for the scalability of human oversight in AI technology include active learning techniques for decision scenarios, tiered human oversight for decision scenarios, and sampling-based audit techniques. Future research on human oversight in AI technology should include the development of more sophisticated techniques for identifying scenarios that require human attention and differentiating them from scenarios that can be processed by machines. The development of such techniques can help AI technology in accurately assessing its own uncertainty and can help in the effective allocation of human resources for decision scenarios. The development of different human–AI team configurations can help in identifying different human–AI team configurations that can help in the effective allocation of human resources for decision scenarios. The development of AI technology that can explain itself and identify potential issues can help human overseers in effectively allocating resources for decision scenarios.人类监督的可扩展性是有效实施“人在回路”（HITL）技术的关键问题之一。AI 技术越是扩展到不同的决策场景和环境中，人类的监督能力就越受限。目前针对 AI 技术中人类监督可扩展性的解决方案包括：针对决策场景的主动学习技术、针对决策场景的分级人类监督，以及基于抽样的审计技术。未来关于 AI 技术中人类监督的研究应包括开发更复杂的技术，以识别需要人类关注的场景，并将其与机器可处理的场景区分开来。开发此类技术有助于 AI 技术准确评估自身的不确定性，并有助于在决策场景中有效地分配人力资源。开发不同的人机团队配置，有助于确定哪些配置能够帮助在决策场景中有效地分配人力资源。开发能够自我解释并识别潜在问题的 AI 技术，可以帮助人类监督者在决策场景中有效地分配资源。

8.3. Human Factors and Cognitive Limitations8.3. 人为因素与认知局限性

The cognitive limitations of humans are fundamental constraints on the performance of human-in-the-loop systems that cannot be alleviated by technology alone. Fatigue, lapses of attention, cognitive biases, and the bounded nature of human rationality are all factors that can affect the performance of humans in an oversight role. These factors may be exacerbated by the characteristics of the HITL tasks, such as the need to maintain vigilance, repetitive tasks, and the difficulty of maintaining engagement when the AI system is working well. Research on the human factors associated with HITL systems has identified the issues associated with the performance of humans in an oversight role. However, the solution to the problem remains an open issue.人类的认知局限性是“人在回路”系统性能的基础性制约因素，仅靠技术无法缓解。疲劳、注意力缺失、认知偏差以及人类理性的有限性，都是影响人类在监督角色中表现的因素。HITL 任务的特性（如需要保持警觉、重复性任务，以及在 AI 系统运行良好时难以保持投入）可能会加剧这些因素的影响。关于 HITL 系统相关人为因素的研究已经识别出与人类在监督角色中表现相关的问题，但如何解决这一问题仍是一个悬而未决的课题。

The areas to be addressed in future research on HITL systems are the design of the system to accommodate the limitations of humans and the development of support tools to enhance the performance of humans. Adaptive systems that can respond to the cognitive state of humans by making adjustments to the system to accommodate the limitations of humans are a promising area of research. However, the question of what remedial action to take when a problem is detected is still an open one. Training methods to enhance the skills of humans in an HITL system are an important area of research. This is especially true in the face of the problem of the degradation of skills when AI systems are used. The design of work schedules to maintain the performance of humans over long periods of time is an important area of research.未来 HITL 系统研究需要解决的领域包括：设计能够适应人类局限性的系统，以及开发能够增强人类表现的支持工具。能够通过调整系统以适应人类局限性来响应人类认知状态的自适应系统，是一个有前景的研究领域。然而，当检测到问题时应采取何种补救措施，仍是一个悬而未决的问题。旨在提升人类在 HITL 系统中技能的培训方法是一个重要的研究领域。面对使用 AI 系统时可能出现的技能退化问题，这一点尤为重要。设计能够使人类在长时间内保持良好表现的工作时间表，也是一个重要的研究方向。

8.4. Conflicting Human Feedback8.4. 冲突的人类反馈

In human-in-the-loop (HITL) systems that combine the feedback of various contributors, there is a problem of disagreement among the contributors. There is a chance for annotators to assign different labels to similar situations, for stakeholders to hold incompatible views on fairness criteria, and for experts to draw different conclusions on the best course of action. The existing solutions to the problem of dealing with the disagreements of human contributors include majority voting, quality-weighted aggregation of the inputs from the human contributors, and approaches that preserve the information on the disagreements rather than forcing consensus. However, these solutions do not completely address the problem of how to create ground truth in the face of human conflict. The area of conflicting human feedback should be further researched with a differentiation between the cases of human conflict due to ambiguity or plurality of values on the one hand, and human conflict due to errors or lack of information on the other. The development of appropriate tools for detecting the causes of the conflict may help to address the different types of conflict appropriately. Research on the deliberative procedures that help to achieve consensus among humans or that help to understand the nature of the conflict may complement the aggregation-based approaches to human feedback that treat human opinions as given. The impact of conflicting human feedback on the learning of the system and the validity of the AI actions should continue to be an important research question.在结合了多方贡献者反馈的“人在回路”（HITL）系统中，存在贡献者之间意见不一致的问题。标注者可能会为相似的情况分配不同的标签，利益相关者可能对公平性标准持有不相容的观点，专家也可能对最佳行动方案得出不同的结论。现有的处理人类贡献者分歧的解决方案包括：多数投票法、对人类贡献者的输入进行质量加权聚合，以及保留分歧信息而非强求共识的方法。然而，这些解决方案并未完全解决在人类冲突面前如何建立“基准真值”（ground truth）的问题。应进一步研究冲突的人类反馈这一领域，并区分以下两类情况：一类是由于歧义或价值观多元化导致的人类冲突，另一类是由于错误或信息匮乏导致的人类冲突。开发适当的工具来检测冲突原因，可能有助于针对不同类型的冲突采取适当的对策。研究有助于在人类之间达成共识或有助于理解冲突本质的审议程序，可以补充那些将人类意见视为既定事实的基于聚合的反馈方法。冲突的人类反馈对系统学习和 AI 行为有效性的影响，应继续作为一个重要的研究课题。

8.5. Adversarial Manipulation and Security8.5. 对抗性操纵与安全

Human-in-the-loop (HITL) systems incorporate human components that may be vulnerable to adversarial attacks, such as social engineering attacks on trust, cognition, or organizational factors. Attackers may aim to compromise training data by attacking human annotators, evade detection by exploiting human fatigue, or manipulate system behavior by targeting individuals who are involved in feedback provision or deployment decision-making. Hence, the security of HITL systems will not only depend on system security, but also on the robustness of human components against adversarial attacks.“人在回路”（HITL）系统包含可能易受对抗性攻击的人类组件，例如针对信任、认知或组织因素的社会工程攻击。攻击者可能试图通过攻击人类标注者来破坏训练数据，通过利用人类疲劳来逃避检测，或者通过针对参与反馈提供或部署决策的个人来操纵系统行为。因此，HITL 系统的安全性不仅取决于系统安全，还取决于人类组件抵御对抗性攻击的稳健性。

In order to advance HITL security, it is recommended that in the future, threat models for HITL systems are developed, highlighting vulnerabilities in HITL systems resulting from human involvement, as well as proposing countermeasures for different risk contexts. Detection of potential attacks on human components, such as detection of anomalous behavior or feedback, may aid in early detection of attacks on human components in HITL systems. In addition, organizational factors that reduce susceptibility to social engineering attacks should be given more importance in HITL system security, in addition to technical security aspects. Developing HITL systems with security guarantees even when some human components are compromised is a promising yet challenging direction for future research on HITL system security.为了推进 HITL 安全，建议在未来开发 HITL 系统威胁模型，突出因人类参与而产生的系统脆弱性，并针对不同的风险环境提出应对措施。检测针对人类组件的潜在攻击（例如检测异常行为或异常反馈），可能有助于在 HITL 系统中及早发现针对人类组件的攻击。此外，除了技术安全方面，在 HITL 系统安全中还应更加重视降低社会工程攻击易感性的组织因素。开发即使在某些人类组件被破坏时仍能提供安全保证的 HITL 系统，是未来 HITL 系统安全研究中一个充满希望但极具挑战性的方向。

8.6. Toward Adaptive and Self-Regulating Architectures8.6. 迈向自适应与自调节架构

The existing state of affairs of human-in-the-loop systems is based on a set of fixed configurations that identify when human input is required during specific junctures of AI system execution. A system that is adaptive in nature, where human input is modulated based on risk levels, AI confidence, or performance, could be seen as offering a better balance between levels of oversight and system efficiency. However, a self-regulating system also poses a unique problem: if left to its own devices, where it is responsible for determining when levels of oversight are required, it is possible that problems of self-assessment could result in levels of oversight being minimized or eliminated altogether. This creates a vicious loop that undermines the normative purpose of a HITL system unless bounded by external limits that are not represented within the model that is being overseen.现有“人在回路”系统的现状基于一套固定的配置，这些配置用于识别 AI 系统执行过程中的特定环节何时需要人类输入。一种本质上具有自适应性的系统，即根据风险水平、AI 置信度或性能来调节人类输入，可以被视为在监督水平与系统效率之间提供了更好的平衡。然而，自调节系统也提出了一个独特的问题：如果任其自行决定何时需要监督，那么由于自我评估问题，可能会导致监督水平被最小化甚至完全消除。除非受到模型本身所未表示的外部限制的约束，否则这会形成一个破坏 HITL 系统规范性目的的恶性循环。

A solution that is robust in nature would be to ensure that adaptive triggering is supplemented by non-adaptive measures of oversight. These could be represented by measures that are based on hard levels of oversight for high levels of risk, policy-based intervention rules that are developed by external agencies or organizations, randomized mandatory audit rules, sentinel models that are independent of the AI system, or default levels of oversight that are conservative in nature when uncertainty or changes in probability are represented by levels that are unstable or uncertain. In this regard, risk levels determined by AI represent a means of identifying where levels of oversight are required but do not represent a means of eliminating levels of oversight altogether. Future research should focus on ensuring that meta-oversight is incorporated, where humans govern the levels of adaptation in AI systems, including trigger levels, override levels, and post-incident levels of assessment.一个稳健的解决方案是确保自适应触发机制辅以非自适应的监督措施。这些措施可以包括：针对高风险水平的硬性监督措施、由外部机构或组织制定的基于政策的干预规则、随机强制审计规则、独立于 AI 系统的哨兵模型，或者在不确定性或概率变化表现为不稳定或不确定水平时采取保守的默认监督水平。在这方面，AI 确定的风险水平提供了一种识别何处需要监督的方法，但并不意味着可以完全消除监督。未来的研究应侧重于确保纳入元监督（meta-oversight），即由人类来管理 AI 系统中的适应水平，包括触发水平、覆盖水平和事后评估水平。