arXiv is now an independent nonprofit! Learn more
License: CC BY 4.0
arXiv:2608.26263v2 [cs.AI] 28 Aug 2026

SKILL.state: Scalable Long-Horizon Agent SkillsSKILL.state:可扩展的长程智能体技能

Sanket Badhe Affiliation: Google LLC    Priyanka Tiwari, Jonghyun Chung Affiliation: Google LLC Affiliation: Purdue University Correspondence:sanketbadhe@google.com
Abstract摘要

Large Language Models (LLMs) increasingly act as autonomous agents executing complex, long-running procedural skills. Existing agent runtimes maintain execution by continually appending observations, actions, and intermediate reasoning traces to an ever-growing conversation history, causing latency degradation and context-poisoning failures over long horizons. We present SKILL.state, a runtime architecture that replaces append-only conversational history with an explicit, mutable execution state. At each execution step, the model receives only the immutable skill specification, the current structured execution state, and the latest observation. Intermediate reasoning is discarded immediately after producing a validated state update, preventing prompt growth with execution history. Across diverse datasets, models, and execution environments, SKILL.state improves task accuracy while substantially reducing cumulative token consumption. Our results demonstrate that explicit execution state is an effective and architecture-agnostic abstraction for scalable long-horizon agent skills.大型语言模型(LLM)正越来越多地作为自主智能体执行复杂、长周期的程序化技能。现有的智能体运行时通过不断将观察结果、动作和中间推理轨迹附加到日益增长的对话历史中来维持执行,这会导致长程任务中的延迟增加和上下文污染问题。我们提出了 SKILL.state,这是一种运行时架构,它用显式的、可变的执行状态取代了仅能追加的对话历史。在每个执行步骤中,模型仅接收不可变的技能规范、当前的结构化执行状态以及最新的观察结果。中间推理在产生经过验证的状态更新后立即被丢弃,从而避免了提示词随执行历史而增长。在各种数据集、模型和执行环境中,SKILL.state 在提高任务准确性的同时,大幅降低了累积的 Token 消耗。我们的结果表明,显式执行状态是一种有效且与架构无关的抽象,适用于可扩展的长程智能体技能。

1 Introduction1 引言

Large Language Models (LLMs) have rapidly evolved from passive language interfaces into autonomous systems capable of iterative reasoning, tool use, and interaction with external environments (Yao et al., 2022; Schick et al., 2023; Qin et al., 2024; Wu et al., 2023). Recent work further demonstrates that these capabilities can be encapsulated as reusable procedural skills, enabling agents to perform software engineering, workflow automation, web interaction, and scientific discovery through modular compositions of specialized behaviors (Badhe et al., 2026). As agents increasingly execute long-running procedures, execution itself becomes a systems problem rather than purely a reasoning problem.大型语言模型(LLM)已迅速从被动的语言接口演变为能够进行迭代推理、工具使用以及与外部环境交互的自主系统(Yao et al., 2022; Schick et al., 2023; Qin et al., 2024; Wu et al., 2023)。近期研究进一步表明,这些能力可以封装为可重用的程序化技能,使智能体能够通过专门行为的模块化组合来执行软件工程、工作流自动化、网页交互和科学发现(Badhe et al., 2026)。随着智能体越来越多地执行长周期程序,执行本身已成为一个系统问题,而不仅仅是一个推理问题。

Modern agent runtimes almost universally adopt a conversational execution model. At every execution step, the language model receives the original skill specification together with an ever-growing transcript of previous reasoning, actions, observations, and tool outputs (Yao et al., 2022; Mialon et al., 2023). Although memory systems alleviate context growth through summarization or retrieval (Packer et al., 2023; Wang et al., 2023; Zhong et al., 2023), they preserve the same execution semantics: future decisions are conditioned on textual reconstructions of past execution rather than an explicit representation of the current execution state.现代智能体运行时几乎普遍采用对话式执行模型。在每个执行步骤中,语言模型都会接收原始技能规范以及一份不断增长的、包含先前推理、动作、观察和工具输出的记录(Yao et al., 2022; Mialon et al., 2023)。尽管内存系统通过摘要或检索(Packer et al., 2023; Wang et al., 2023; Zhong et al., 2023)缓解了上下文增长问题,但它们保留了相同的执行语义:未来的决策基于对过去执行的文本重构,而不是基于当前执行状态的显式表示。

This design introduces fundamental limitations for long-horizon procedural skills. Prompt size grows with execution length, increasing token consumption and inference cost (Liu et al., 2024; Xiao et al., 2024a). Historical observations and obsolete reasoning remain embedded in the context long after they cease to be relevant, requiring the model to continually distinguish current facts from historical artifacts. Consequently, execution correctness increasingly depends on reconstructing state from accumulated textual history.这种设计为长程程序化技能带来了根本性的局限。提示词大小随执行长度增加,导致 Token 消耗和推理成本上升(Liu et al., 2024; Xiao et al., 2024a)。历史观察和过时的推理在失去相关性后仍留在上下文中,迫使模型不断地从历史伪影中区分当前事实。因此,执行的正确性越来越依赖于从累积的文本历史中重构状态。

In this paper, we introduce SKILL.state, a runtime architecture that reformulates procedural skill execution as explicit state transitions rather than conversational history accumulation. Figure 1 provides an overview of the proposed runtime. At execution step tt, the language model receives only three inputs:在本文中,我们引入了 SKILL.state,这是一种将程序化技能执行重构为显式状态转换而非对话历史累积的运行时架构。图 1 提供了所提运行时的概览。在执行步骤 t,语言模型仅接收三个输入:

At=(P,Σt,Ot),A_{t}=(P,\Sigma_{t},O_{t}), (1)

where PP denotes the immutable procedural specification, Σt\Sigma_{t} is the structured execution state, and OtO_{t} is the latest environment observation. After producing a validated state update, the intermediate reasoning trace is discarded while only the updated execution state is retained. Consequently, execution depends strictly on the current world state instead of replaying historical trajectories.其中 P 表示不可变的程序规范,Σt 表示结构化执行状态,Ot 表示最新的环境观察结果。在产生经过验证的状态更新后,中间推理轨迹被丢弃,仅保留更新后的执行状态。因此,执行严格依赖于当前的世界状态,而不是重放历史轨迹。

To evaluate this hypothesis, we evaluate SKILL.state across both synthetic and real-world benchmarks: SkillExecBench, a controlled benchmark designed for long-horizon procedural skill execution under scaling, noise, and state recovery; InterCode CTF (Yang et al., 2023), featuring interactive Linux terminal exploitation; and Sierra τ\tau-Bench (Yao et al., 2024), evaluating multi-turn customer-service workflows over complex database APIs.为了验证这一假设,我们在合成基准和现实世界基准上评估了 SKILL.state:SkillExecBench(一个受控基准,旨在测试缩放、噪声和状态恢复下的长程程序化技能执行)、InterCode CTF(Yang et al., 2023,具有交互式 Linux 终端利用功能)以及 Sierra τ-Bench(Yao et al., 2024,评估复杂数据库 API 上的多轮客户服务工作流)。

Experimental results demonstrate that explicit execution state substantially improves the scalability of long-horizon procedural skills by maintaining bounded prompt sizes while cutting token consumption and outperforming history-based and compression-based baselines across multiple model families.实验结果表明,显式执行状态通过保持有界的提示词大小,同时减少 Token 消耗,并在多个模型系列中优于基于历史和基于压缩的基线,从而显著提高了长程程序化技能的可扩展性。

Our contributions are summarized as follows:我们的贡献总结如下:

  • •

    We propose SKILL.state, a runtime architecture that executes procedural skills through explicit structured execution state where intermediate reasoning is discarded after each step, proving a strictly bounded 𝒪⁡(1)\mathcal{O}(1) prompt footprint and 𝒪⁡(T)\mathcal{O}(T) cumulative token complexity.我们提出了 SKILL.state,这是一种通过显式结构化执行状态执行程序化技能的运行时架构,其中中间推理在每一步后被丢弃,从而证明了严格有界的 O(1) 提示词占用空间和 O(T) 的累积 Token 复杂度。

  • •

    We present SkillExecBench, alongside evaluations on public benchmarks (InterCode CTF and Sierra τ\tau-Bench), for evaluating long-horizon procedural skill execution in sequential, stateful environments.我们提出了 SkillExecBench,并结合公共基准(InterCode CTF 和 Sierra τ-Bench)的评估,用于评估顺序、有状态环境中的长程程序化技能执行。

  • •

    Across multiple execution horizons and runtime baselines, we demonstrate that state-centric execution maintains competitive task performance while substantially reducing prompt growth and cumulative token consumption across both proprietary and open-weight models.在多个执行周期和运行时基线上,我们证明了以状态为中心的执行在保持竞争性任务性能的同时,显著降低了专有模型和开源权重模型中的提示词增长和累积 Token 消耗。

2 Related Work2 相关工作

2.1 Procedural Skills for LLM Agents2.1 LLM 智能体的程序化技能

Existing research on reusable procedural skills primarily addresses skill discovery, representation, composition and and security threat modeling (Badhe et al., 2026; Badhe and Tiwari, 2026). Our work instead focuses on the largely unexplored mechanics of skill execution once a skill has been selected.现有的关于可重用程序化技能的研究主要探讨技能发现、表示、组合和安全威胁建模(Badhe et al., 2026; Badhe and Tiwari, 2026)。我们的工作则专注于技能一旦被选中后,在很大程度上未被探索的技能执行机制。

2.2 Memory Architectures for Long-Horizon Agents2.2 长程智能体的内存架构

Long-horizon agent architectures typically preserve conversational semantics through episodic retrieval (Park et al., 2023) or persistent storage (Chhikara et al., 2025; Zhong et al., 2024). These methods leave execution state implicitly distributed across accumulated logs. SKILL.state instead isolates execution into an explicit, mutable runtime state, eliminating the need to repeatedly reconstruct world models from textual history. Frameworks like LangGraph use auxiliary structured state to orchestrate workflows across agent nodes. However, these systems still rely on conversational transcripts as the primary reasoning substrate. SKILL.state replaces this substrate by discarding intermediate reasoning traces immediately after producing validated state transitions.长程智能体架构通常通过情景检索(Park et al., 2023)或持久存储(Chhikara et al., 2025; Zhong et al., 2024)来维持对话语义。这些方法使执行状态隐式地分布在累积的日志中。SKILL.state 则将执行隔离为显式的、可变的运行时状态,消除了从文本历史中重复重构世界模型的需要。像 LangGraph 这样的框架使用辅助结构化状态来协调跨智能体节点的工作流。然而,这些系统仍然依赖对话记录作为主要的推理基质。SKILL.state 通过在产生经过验证的状态转换后立即丢弃中间推理轨迹来替换这一基质。

2.3 Dialogue State Tracking2.3 对话状态跟踪

Dialogue State Tracking (DST) maintains user slot values across conversational turns in task-oriented dialogue (Williams et al., 2013; Henderson et al., 2014; Rastogi et al., 2020; Wu et al., 2019; Heck et al., 2020; Hosseini-Asl et al., 2020). While both DST and SKILL.state maintain structured representations, they differ fundamentally in execution mechanics: DST tracks auxiliary state alongside full conversational transcripts in quasi-static dialogues, whereas SKILL.state treats the structured state as a sufficient statistic, discarding conversational history to execute autonomous skills in dynamic environments with bounded prompt footprints.对话状态跟踪(DST)在任务导向型对话中跨对话轮次维护用户槽位值(Williams et al., 2013; Henderson et al., 2014; Rastogi et al., 2020; Wu et al., 2019; Heck et al., 2020; Hosseini-Asl et al., 2020)。虽然 DST 和 SKILL.state 都维护结构化表示,但它们在执行机制上有根本区别:DST 在准静态对话中跟踪辅助状态以及完整的对话记录,而 SKILL.state 将结构化状态视为充分统计量,丢弃对话历史,从而在具有有界提示词占用空间的动态环境中执行自主技能。

2.4 Context Management and Long-Context Reasoning2.4 上下文管理与长上下文推理

Language models exhibit degraded retrieval over long contexts (Liu et al., 2024; Zhang et al., 2024), motivating streaming attention (Xiao et al., 2024b) and prompt compression techniques (Jiang et al., 2023; Li et al., 2023). Rather than attempting to process or compress extended conversational histories, SKILL.state prevents history accumulation entirely by maintaining the canonical execution state required for the next computation.语言模型在长上下文中表现出检索性能下降(Liu et al., 2024; Zhang et al., 2024),这促使了流式注意力(Xiao et al., 2024b)和提示词压缩技术(Jiang et al., 2023; Li et al., 2023)的发展。SKILL.state 不是尝试处理或压缩扩展的对话历史,而是通过维护下一次计算所需的规范执行状态,完全防止了历史累积。

3 SKILL.state3 SKILL.state

Current LLM agent runtimes execute procedural skills by repeatedly appending reasoning traces, actions, observations, and tool outputs to a growing conversational history. Consequently, the execution state is represented implicitly within natural language and must be reconstructed by the language model at every interaction. As execution horizons increase, both prompt size and the volume of obsolete information grow monotonically, making execution increasingly dependent on interpreting historical text rather than maintaining the current world state.当前的 LLM 智能体运行时通过重复将推理轨迹、动作、观察和工具输出附加到不断增长的对话历史中来执行程序化技能。因此,执行状态隐式地表示在自然语言中,必须由语言模型在每次交互时进行重构。随着执行周期的增加,提示词大小和过时信息的量单调增长,使得执行越来越依赖于解释历史文本,而不是维护当前的世界状态。

SKILL.state reformulates procedural skill execution as an explicit state transition process. Instead of representing execution as an append-only conversation, every execution step is defined by:SKILL.state 将程序化技能执行重构为显式的状态转换过程。每个执行步骤不是表示为仅能追加的对话,而是定义为:

At=(P,Σt,Ot),A_{t}=(P,\Sigma_{t},O_{t}), (2)

where PP is the immutable procedural specification, Σt\Sigma_{t} is the structured execution state at step tt, and OtO_{t} is the latest observation received from the environment. The language model never receives previous observations, previous actions, or previous reasoning traces.其中 P 是不可变的程序规范,Σt 是步骤 t 的结构化执行状态,Ot 是从环境中接收到的最新观察结果。语言模型永远不会接收先前的观察结果、先前的动作或先前的推理轨迹。

Figure 1 illustrates the execution cycle. At each step, the runtime constructs a prompt from (P,Σt,Ot)(P,\Sigma_{t},O_{t}), invokes the language model, deterministically validates the proposed state transition, updates the execution state, executes the selected action, and repeats the process using the updated state.图 1 展示了执行周期。在每一步中,运行时根据 (P, Σt, Ot) 构建提示词,调用语言模型,确定性地验证建议的状态转换,更新执行状态,执行选定的动作,并使用更新后的状态重复该过程。

Figure 1: Overview of the SKILL.state architecture.图 1:SKILL.state 架构概览。

3.1 Execution State and Schema Authoring3.1 执行状态与模式创作

Unlike conversational runtimes, SKILL.state treats execution state as a first-class runtime abstraction. The state contains only information required for future execution and is represented using a structured schema defined for the domain. Schemas are authored once per domain rather than per task; for example, across all 100 diverse challenge instances in the InterCode CTF benchmark, the agent reuses a single static 5-field schema (discovered_flags, tested_hypotheses, active_files, working_dir, cmd_summary).与对话式运行时不同,SKILL.state 将执行状态视为一等运行时抽象。状态仅包含未来执行所需的信息,并使用为该领域定义的结构化模式表示。模式是每个领域创作一次,而不是每个任务创作一次;例如,在 InterCode CTF 基准的所有 100 个不同挑战实例中,智能体重用了一个静态的 5 字段模式(discovered_flags, tested_hypotheses, active_files, working_dir, cmd_summary)。

3.2 Reasoning and State Transitions3.2 推理与状态转换

Reasoning is used strictly as an intermediate computation for producing state transitions and selecting the next action. Given the current execution context (P,Σt,Ot)(P,\Sigma_{t},O_{t}), the language model generates:推理严格用作产生状态转换和选择下一个动作的中间计算。给定当前的执行上下文 (P, Σt, Ot),语言模型生成:

(Rt,Δ​Σt,at),(R_{t},\Delta\Sigma_{t},a_{t}), (3)

where RtR_{t} denotes the multi-step Chain-of-Thought reasoning trace, Δ​Σt\Delta\Sigma_{t} is a structured state update (a JSON dictionary of key mutations and deletions), and ata_{t} is the action to execute.其中 Rt 表示多步思维链推理轨迹,ΔΣt 是结构化状态更新(键突变和删除的 JSON 字典),at 是要执行的动作。

Crucially, within-step multi-step reasoning is fully intact during generation to support complex deductive planning. However, once the state transition has been validated and applied, the reasoning trace RtR_{t} is discarded permanently and never appears in subsequent prompts. The execution state is updated according to:至关重要的是,步骤内的多步推理在生成过程中保持完整,以支持复杂的演绎规划。然而,一旦状态转换经过验证并应用,推理轨迹 Rt 将被永久丢弃,且永远不会出现在后续的提示词中。执行状态根据以下公式更新:

Σt+1=Σt⊕Δ​Σt,\Sigma_{t+1}=\Sigma_{t}\oplus\Delta\Sigma_{t}, (4)

where ⊕\oplus denotes the runtime’s dictionary merge operator with null-deletion semantics. This model projects transient reasoning into persistent structured state, allowing only information required for future execution to survive across interactions.其中 ⊕ 表示运行时的字典合并运算符,具有空删除语义。该模型将瞬态推理投影到持久的结构化状态中,仅允许未来执行所需的信息在交互中存续。

Algorithm 1 SKILL.state Runtime算法 1 SKILL.state 运行时
1: Procedural specification PP, initial state Σ0\Sigma_{0}
2: forfor t=0,…,Tt=0,\ldots,T do
3:   Receive latest observation OtO_{t}
4:   Construct prompt (P,Σt,Ot)(P,\Sigma_{t},O_{t})
5:   Generate (Rt,Δ​Σt,at)(R_{t},\Delta\Sigma_{t},a_{t}) using the LLM
6:   Validate Δ​Σt\Delta\Sigma_{t}
7:   Σt+1←Σt⊕Δ​Σt\Sigma_{t+1}\leftarrow\Sigma_{t}\oplus\Delta\Sigma_{t}
8:   Execute ata_{t}
9: endend for

Algorithm 1 summarizes the execution process.算法 1 总结了执行过程。

3.3 Complexity Analysis3.3 复杂度分析

Let TT denote the execution horizon. For conversational runtimes, prompt length grows with the accumulated interaction history, |Ct|=𝒪⁡(t)|C_{t}|=\mathcal{O}(t), leading to cumulative token complexity:设 T 表示执行周期。对于对话式运行时,提示词长度随累积的交互历史增长,|Ct|=O(t),导致累积 Token 复杂度为:

∑t=1T|Ct|=𝒪⁡(T2).\sum_{t=1}^{T}|C_{t}|=\mathcal{O}(T^{2}). (5)

In contrast, SKILL.state maintains only the procedural specification, structured execution state, and latest observation:相比之下,SKILL.state 仅维护程序规范、结构化执行状态和最新观察结果:

|Pt|=𝒪⁡(|P|+|Σ|+|O|),|P_{t}|=\mathcal{O}(|P|+|\Sigma|+|O|), (6)

which is asymptotically bounded and independent of the number of previously executed turns tt. Consequently, cumulative prompt complexity grows strictly linearly with the execution horizon:它是渐近有界的,且与先前执行的轮次 t 无关。因此,累积提示词复杂度随执行周期严格线性增长:

∑t=1T|Pt|=𝒪⁡(T).\sum_{t=1}^{T}|P_{t}|=\mathcal{O}(T). (7)

The resulting runtime shifts execution from reconstructing history toward maintaining an explicit, validated representation of the current execution state.由此产生的运行时将执行重心从重构历史转向维护当前执行状态的显式、经过验证的表示。

4 Evaluation Benchmarks4 评估基准

4.1 SkillExecBench (Controlled Diagnostic Testbed)4.1 SkillExecBench(受控诊断测试平台)

SkillExecBench isolates execution mechanics from open-ended heuristic search by providing sequential procedural tasks with deterministic ground-truth world transitions:SkillExecBench 通过提供具有确定性真值世界转换的顺序程序化任务,将执行机制与开放式启发式搜索隔离开来:

  • •

    Environment 1 (Warehouse Management): A discrete physical inventory domain tracking 500 independent shelves. Actions include Store, Ship, Move, and Wait. This environment tests the model’s ability to maintain independent, non-overlapping state variables over extended horizons where early observations leave the context window.环境 1(仓库管理):一个离散的物理库存领域,跟踪 500 个独立货架。动作包括存储、发货、移动和等待。该环境测试模型在早期观察离开上下文窗口的长周期内,维护独立、不重叠的状态变量的能力。

  • •

    Environment 2 (Software Repository): A deeply nested, relational graph of Git branches, commits, Pull Requests, and CI test statuses. Actions include CherryPick, Merge, RunTests, CreateRelease, and Rollback. Features dense dependencies where a single action (e.g., merging a PR) fundamentally alters the state of the target branch and dependent PRs, testing complex structural reasoning over an entangled graph.环境 2(软件仓库):一个深度嵌套的 Git 分支、提交、拉取请求和 CI 测试状态的关系图。动作包括 CherryPick、合并、运行测试、创建发布和回滚。具有密集依赖关系,其中单个动作(例如合并 PR)会从根本上改变目标分支和相关 PR 的状态,测试对纠缠图的复杂结构推理。

4.2 Public Interactive Benchmarks4.2 公共交互式基准

To evaluate SKILL.state on real-world, non-deterministic tasks with complex search, generation, and tool use, we evaluate on two public benchmarks:为了在具有复杂搜索、生成和工具使用的真实、非确定性任务上评估 SKILL.state,我们在两个公共基准上进行了评估:

  • •

    InterCode CTF (Yang et al., 2023): A suite of 100 Linux bash Capture-The-Flag challenges spanning reverse engineering, forensics, cryptography, and binary exploitation. Agents execute bash commands in Docker containers and iteratively test hypotheses to discover hidden flags.InterCode CTF (Yang et al., 2023):一套包含 100 个 Linux bash Capture-The-Flag 挑战的套件,涵盖逆向工程、取证、密码学和二进制利用。智能体在 Docker 容器中执行 bash 命令并迭代测试假设以发现隐藏标志。

  • •

    Sierra τ\tau-Bench (Yao et al., 2024): A benchmark for tool-agent-user interaction in enterprise customer service (Retail and Airline domains). Agents interact with simulated users, query relational SQLite databases via tool calls, and execute transactional actions (e.g., flight rebooking, refunds) under business policy constraints.Sierra τ-Bench (Yao et al., 2024):一个用于企业客户服务(零售和航空领域)中工具-智能体-用户交互的基准。智能体与模拟用户交互,通过工具调用查询关系型 SQLite 数据库,并在业务策略约束下执行事务性动作(例如航班改签、退款)。

4.3 Evaluation Metrics4.3 评估指标

We evaluate runtimes across three dimensions:我们从三个维度评估运行时:

  • •

    Task Accuracy / Success Rate: In SkillExecBench, accuracy is measured continuously as the ratio of valid, correct actions matching the ground-truth deterministic simulation (Score=Successful ActionsTotal Actionable Events\text{Score}=\frac{\text{Successful Actions}}{\text{Total Actionable Events}}). In InterCode CTF, success is binary pass@1 (exact match on the binary-verified flag). In τ\tau-Bench, success is scored by the official programmatic evaluator, which verifies that the final database state satisfies user intent without policy violations.任务准确性/成功率:在 SkillExecBench 中,准确性被持续测量为匹配真值确定性模拟的有效、正确动作的比率(Score=成功动作/总可操作事件)。在 InterCode CTF 中,成功是二进制的 pass@1(二进制验证标志的精确匹配)。在 τ-Bench 中,成功由官方程序化评估器评分,该评估器验证最终数据库状态是否满足用户意图且无策略违规。

  • •

    Average Prompt Size: The mean token footprint per LLM invocation.平均提示词大小:每次 LLM 调用时的平均 Token 占用空间。

  • •

    Total Token Cost: The cumulative token burn across the entire execution horizon.总 Token 成本:整个执行周期的累积 Token 消耗。

Table 1: Warehouse Management Long-Horizon Scaling using Gemini-3-Flash. Baseline runtimes suffer 𝒪⁡(T2)\mathcal{O}(T^{2}) context accumulation, whereas SKILL.state maintains a bounded 𝒪⁡(1)\mathcal{O}(1) prompt footprint (Mean ±\pm SD across 5 seeds).表 1:使用 Gemini-3-Flash 进行仓库管理的长程扩展。基线运行时遭受 O(T^2) 的上下文累积,而 SKILL.state 保持有界的 O(1) 提示词占用空间(5 个种子的平均值 ± 标准差)。
Horizon (TT) Runtime Score (Accuracy) Avg Prompt (Tokens) Total Tokens
10 Prompt (ReAct) 0.90 ±\pm0.02 3,249 ±\pm94 9,438 ±\pm371
Memory (Summary) 1.00 ±\pm0.00 3,300 ±\pm123 9,972 ±\pm204
Stateful (LangGraph) 1.00 ±\pm0.00 3,430 ±\pm42 10,337 ±\pm299
SKILL.state 1.00 ±\pm0.00 1,775 ±\pm74 5,870 ±\pm131
25 Prompt (ReAct) 0.92 ±\pm0.02 6,052 ±\pm192 42,689 ±\pm2,238
Memory (Summary) 0.99 ±\pm0.00 6,357 ±\pm203 43,067 ±\pm1,948
Stateful (LangGraph) 1.00 ±\pm0.00 5,858 ±\pm301 41,238 ±\pm3,196
SKILL.state 1.00 ±\pm0.00 1,736 ±\pm49 14,714 ±\pm564
50 Prompt (ReAct) 0.88 ±\pm0.04 11,931 ±\pm346 171,658 ±\pm6,978
Memory (Summary) 0.93 ±\pm0.03 7,582 ±\pm283 131,455 ±\pm6,841
Stateful (LangGraph) 0.94 ±\pm0.00 11,594 ±\pm438 170,992 ±\pm7,918
SKILL.state 0.96 ±\pm0.01 1,773 ±\pm53 30,151 ±\pm1,231
100 Prompt (ReAct) 0.84 ±\pm0.07 36,362 ±\pm1,304 1,245,413 ±\pm53,241
Memory (Summary) 0.87 ±\pm0.05 29,607 ±\pm978 1,082,154 ±\pm83,212
Stateful (LangGraph) 0.91 ±\pm0.02 31,354 ±\pm831 1,062,387 ±\pm53,839
SKILL.state 0.94 ±\pm0.01 1,905 ±\pm93 65,408 ±\pm5,431
200 Prompt (ReAct) 0.74 ±\pm0.14 48,007 ±\pm2,092 2,608,755 ±\pm102,415
Memory (Summary) 0.84 ±\pm0.09 84,364 ±\pm3,446 6,175,509 ±\pm294,089
Stateful (LangGraph) 0.88 ±\pm0.03 72,305 ±\pm3,096 5,041,164 ±\pm346,925
SKILL.state 0.94 ±\pm0.02 1,811 ±\pm184 122,384 ±\pm4,522

5 Experiments and Results5 实验与结果

5.1 Experimental Setup5.1 实验设置

We evaluate SKILL.state against two families of baselines (see Appendix Appendix A. Runtime Prompts for exact prompt templates):我们针对两类基线评估 SKILL.state(确切的提示词模板见附录 A. 运行时提示词):

Primary Runtime Paradigms:主要运行时范式:

  1. 1.

    Prompt (ReAct-style): Appends every observation, intermediate reasoning trace, and action to a continually growing transcript (Yao et al., 2022).提示词(ReAct 风格):将每个观察结果、中间推理轨迹和动作附加到不断增长的记录中(Yao et al., 2022)。

  2. 2.

    Memory (Summarization-style): Maintains a rolling 3-step conversational window alongside a periodically updated natural language summary of past interactions (Packer et al., 2023).内存(摘要风格):维护一个滚动的 3 步对话窗口,以及一个定期更新的过去交互自然语言摘要(Packer et al., 2023)。

  3. 3.

    Stateful (LangGraph-style): Injects a structured state block into the context window alongside the full rolling conversational transcript.有状态(LangGraph 风格):在上下文窗口中注入一个结构化状态块,并伴随完整的滚动对话记录。

Budget-Matched and Compression Controls:预算匹配和压缩控制:

  1. 1.

    Truncated (Sliding Window): Retains only the most recent interaction turns that fit within a fixed token budget.截断(滑动窗口):仅保留适合固定 Token 预算的最近交互轮次。

  2. 2.

    Summary-capped: Strictly enforces a hard token ceiling on the natural language summary.摘要上限:严格强制对自然语言摘要设置硬性 Token 上限。

  3. 3.

    ReAct + LLMLingua (Jiang et al., 2023): Uses budget-aware small-model perplexity compression to prune tokens from the full history down to the target budget.ReAct + LLMLingua (Jiang et al., 2023):使用预算感知的小模型困惑度压缩,将完整历史记录中的 Token 修剪至目标预算。

Underlying Models: Evaluations are conducted across proprietary and open-weight models: Gemini-3-Flash, Gemma-4-31B-it, and Qwen-3-8B-it. Decoding is controlled at temperature 0.00.0 and top-pp 1.01.0 across all runs to ensure deterministic reproducibility.底层模型:评估在专有模型和开源权重模型上进行:Gemini-3-Flash、Gemma-4-31B-it 和 Qwen-3-8B-it。所有运行的解码温度控制在 0.0,top-p 控制在 1.0,以确保确定性的可重复性。

Statistical Significance: All synthetic experiments are evaluated across 5 distinct procedural generator seeds. Results are reported as mean ±\pm sample standard deviation. Differences between SKILL.state and baselines at extended horizons (T≥50T\geq 50) are statistically significant (paired tt-test, p<0.01p<0.01).统计显著性:所有合成实验均在 5 个不同的程序化生成器种子下进行评估。结果报告为平均值 ± 样本标准差。SKILL.state 与基线在扩展周期(T≥50)下的差异具有统计学显著性(配对 t 检验,p<0.01)。

5.2 Experiment 1: Long-Horizon Execution Scaling5.2 实验 1:长程执行扩展

We evaluate runtime accuracy and context expansion across execution horizons scaling from T=10T=10 to T=200T=200 steps.我们评估了执行周期从 T=10 扩展到 T=200 步时的运行时准确性和上下文扩展。

Results: As shown in Table 1, SKILL.state matches or exceeds baseline accuracy across all horizons while maintaining a flat prompt size (∼\sim1,736–1,905 tokens). In contrast, history-appending baselines suffer quadratic token accumulation 𝒪⁡(T2)\mathcal{O}(T^{2}). At T=100T=100, the Stateful baseline consumes 1,062,387 tokens, whereas SKILL.state consumes only 65,408 tokens (a 16.2×16.2\times token reduction). At T=200T=200, SKILL.state maintains 0.94 accuracy consuming 122k tokens, while the Memory baseline inflates to 6.1M tokens. Additional scaling results for the Software Repository and open-weight models are detailed in Appendix Appendix D. Additional Results.结果:如表 1 所示,SKILL.state 在所有周期内匹配或超过了基线准确性,同时保持平坦的提示词大小(约 1,736–1,905 个 Token)。相比之下,历史附加基线遭受二次 Token 累积 O(T^2)。在 T=100 时,有状态基线消耗 1,062,387 个 Token,而 SKILL.state 仅消耗 65,408 个 Token(Token 减少 16.2 倍)。在 T=200 时,SKILL.state 保持 0.94 的准确率,消耗 122k 个 Token,而内存基线膨胀到 6.1M 个 Token。软件仓库和开源权重模型的额外扩展结果详见附录 D. 额外结果。

Table 2: Warehouse Noise Robustness (T=50T=50, Gemini-3-Flash).表 2:仓库噪声鲁棒性(T=50,Gemini-3-Flash)。
Noise Level Runtime Score
5 Events (Low) Prompt 0.68
Memory 1.00
Stateful 1.00
SKILL.state 1.00
20 Events (Medium) Prompt 0.61
Memory 1.00
Stateful 0.98
SKILL.state 0.97
50 Events (High) Prompt 0.53
Memory 0.96
Stateful 0.98
SKILL.state 0.98

5.3 Experiment 2: Context Corruption (Noise Robustness)5.3 实验 2:上下文损坏(噪声鲁棒性)

Real-world execution environments emit dense background telemetry. We fix the horizon at T=50T=50 and inject distractor events (system telemetry, irrelevant git branch activities, and rule overrides) at rates of 5, 20, and 50 events per turn (see Appendix Appendix C. Noise Construction (Experiment 2) for calibration details).现实世界的执行环境会发出密集的背景遥测。我们将周期固定在 T=50,并以每轮 5、20 和 50 个事件的比率注入干扰事件(系统遥测、不相关的 git 分支活动和规则覆盖)(校准细节见附录 C. 噪声构建(实验 2))。

Results: As shown in Table 2, the standard Prompt runtime degrades sharply from 0.68 at low noise down to 0.53 at high noise. In contrast, SKILL.state maintains robust task completion (≥0.97\geq 0.97) across all noise levels because distractors are filtered out during state patch generation and never enter subsequent prompts.结果:如表 2 所示,标准提示词运行时的准确性从低噪声下的 0.68 急剧下降到高噪声下的 0.53。相比之下,SKILL.state 在所有噪声水平下均保持稳健的任务完成度(≥0.97),因为干扰项在状态补丁生成过程中被过滤掉,且从未进入后续提示词。

Table 3: Warehouse State Recovery (Gemini-3-Flash).表 3:仓库状态恢复(Gemini-3-Flash)。
Scenario Runtime Success Recovery Steps
A: Secret Audit Prompt / Memory / Stateful Yes 5–8
SKILL.state Yes 0
B: Secret Barcode Prompt / Memory / Stateful Yes 6–8
SKILL.state Yes 0
C: Secret Move Prompt / Memory / Stateful Yes 5–8
SKILL.state Yes 0
D: Canceled Order All Runtimes No N/A

5.4 Experiment 3: State Recovery5.4 实验 3:状态恢复

We test runtime resilience to silent external environment drift where the true world state is modified outside the agent’s action loop (e.g., an external actor moves an inventory item).我们测试了运行时对静默外部环境漂移的弹性,即真实世界状态在智能体动作循环之外被修改(例如外部参与者移动了库存物品)。

Results: As shown in Table 3, history-based baselines hallucinate for 5 to 8 consecutive turns because obsolete facts in their prompt history overpower contradictory new observations. In sharp contrast, SKILL.state requires zero recovery steps: because its decisions depend on the current structured state, the state is updated immediately upon receiving the corrective alert.结果:如表 3 所示,基于历史的基线产生 5 到 8 个连续轮次的幻觉,因为其提示词历史中的过时事实压倒了矛盾的新观察结果。与此形成鲜明对比的是,SKILL.state 需要零恢复步骤:因为其决策依赖于当前的结构化状态,状态在收到纠正警报后立即更新。

Table 4: Evaluation on Public Interactive Benchmarks using Gemini-3-Flash. SKILL.state achieves the highest task success rates while significantly reducing prompt sizes and cumulative token consumption.表 4:使用 Gemini-3-Flash 在公共交互式基准上的评估。SKILL.state 实现了最高的任务成功率,同时显著减少了提示词大小和累积 Token 消耗。
InterCode CTF (100 Tasks) Sierra τ\tau-Bench (Retail) Sierra τ\tau-Bench (Airline)
Runtime Pass@1 Prompt Tokens Pass Rate Prompt Tokens Pass Rate Prompt Tokens
Prompt (ReAct) 43.2% 1,909 977k 48.2% 2,819 4.48M 21.8% 5,100 4.85M
Memory (Summary) 46.4% 1,797 1.03M 29.9% 2,737 4.24M 23.6% 4,700 4.65M
Stateful (LangGraph) 41.8% 1,946 1.13M 51.7% 3,065 3.92M 28.1% 5,400 5.28M
SKILL.state 54.2% 813 387k 58.3% 3,325 3.47M 32.4% 2,800 2.88M

5.5 Experiment 4: Public Interactive Benchmarks5.5 实验 4:公共交互式基准

To test generalizability on open-ended tasks with complex search, generation, and tool use, we evaluate SKILL.state on InterCode CTF and Sierra τ\tau-Bench.为了测试在具有复杂搜索、生成和工具使用的开放式任务上的通用性,我们在 InterCode CTF 和 Sierra τ-Bench 上评估了 SKILL.state。

Results: As shown in Table 4, SKILL.state achieves the highest task completion rates across all three benchmarks while substantially cutting cumulative token consumption. In InterCode CTF, maintaining explicit hypotheses and discovered flags in Σt\Sigma_{t} prevents the model from repeating failed commands, increasing pass@1 to 54.2% (+7.8 points over the strongest baseline and +12.4 points over Stateful) while cutting total tokens by 60.4% vs. ReAct and 65.9% vs. Stateful. In τ\tau-Bench Retail, SKILL.state leads with 58.3% pass rate at the lowest total token cost. In τ\tau-Bench Airline, where complex database responses cause baseline prompts to peak above 11,000 tokens/step, SKILL.state maintains a flat footprint of ∼\sim2,800 tokens/step and achieves a 32.4% pass rate, saving 40.5% tokens vs. ReAct and 45.4% vs. Stateful.结果:如表 4 所示,SKILL.state 在所有三个基准上均实现了最高的任务完成率,同时大幅降低了累积 Token 消耗。在 InterCode CTF 中,在 Σt 中维护显式假设和已发现标志,防止了模型重复失败的命令,将 pass@1 提高到 54.2%(比最强基线高 7.8 个百分点,比有状态基线高 12.4 个百分点),同时比 ReAct 减少 60.4% 的总 Token,比有状态基线减少 65.9%。在 τ-Bench Retail 中,SKILL.state 以 58.3% 的通过率领先,且总 Token 成本最低。在 τ-Bench Airline 中,复杂的数据库响应导致基线提示词在每步超过 11,000 个 Token,而 SKILL.state 保持约 2,800 个 Token/步的平坦占用空间,并实现了 32.4% 的通过率,比 ReAct 节省 40.5% 的 Token,比有状态基线节省 45.4%。

Table 5: Budget-Matched Controls on Warehouse (T=100T=100, Gemini-3-Flash, Budget ∼\sim1,800 tokens).表 5:仓库上的预算匹配控制(T=100,Gemini-3-Flash,预算约 1,800 个 Token)。
Runtime / Configuration Score Avg Prompt Total Tokens
Full ReAct (Unbounded) 0.84 36,362 1,245,413
Truncated (Sliding Window) 0.18 1,800 62,100
Summary-capped 0.52 1,840 63,400
ReAct + LLMLingua 0.22 1,810 62,350
SKILL.state (Structured) 0.94 1,905 65,408

5.6 Experiment 5: Budget-Matched Controls and Statistical Compression5.6 实验 5:预算匹配控制与统计压缩

To determine whether SKILL.state’s performance gains stem merely from shorter prompts or from structured state representation, we evaluate budget-matched baselines on Warehouse (T=100T=100, Gemini-3-Flash) pinned to the token budget of SKILL.state (∼\sim1,800 tokens). Full multi-horizon scaling across T∈{10,25,50,100}T\in\{10,25,50,100\} is detailed in Appendix .2.为了确定 SKILL.state 的性能提升仅仅源于更短的提示词,还是源于结构化状态表示,我们在 Warehouse (T=100, Gemini-3-Flash) 上评估了预算匹配基线,固定为 SKILL.state 的 Token 预算(约 1,800 个 Token)。跨 T∈{10,25,50,100} 的完整多周期扩展详见附录 .2。

Results: As shown in Table 5, all budget-matched compression baselines suffer catastrophic failure. Sliding-window truncation drops to 0.18 because critical early inventory allocations are evicted. LLMLingua drops to 0.22 because statistical entropy filtering removes seemingly redundant slot identifiers that are semantically vital. In contrast, SKILL.state achieves 0.94 score, demonstrating that structured state maintenance preserves exact relational dependencies that statistical compressors destroy.结果:如表 5 所示,所有预算匹配压缩基线均遭受灾难性失败。滑动窗口截断下降到 0.18,因为关键的早期库存分配被驱逐。LLMLingua 下降到 0.22,因为统计熵过滤移除了语义上至关重要的看似冗余的槽位标识符。相比之下,SKILL.state 实现了 0.94 的分数,证明了结构化状态维护保留了统计压缩器所破坏的精确关系依赖。

5.7 Error Taxonomy for Open-Weight Models5.7 开源权重模型的错误分类

On open-weight models (Gemma-4-31B at T=100T=100, score 0.42), we analyze failure logs and categorize errors into three distinct modes:在开源权重模型(Gemma-4-31B,T=100,得分 0.42)上,我们分析了故障日志并将错误分类为三种不同的模式:

  1. 1.

    Premature State Overwrite / Deletion (68%): The model accidentally omits existing keys during state update rather than merging in-place.过早的状态覆盖/删除 (68%):模型在状态更新期间意外省略了现有键,而不是就地合并。

  2. 2.

    Schema Comprehension / Type Coercion (20%): Inconsistencies between expected nested lists and dictionaries.模式理解/类型强制 (20%):预期嵌套列表和字典之间的不一致。

  3. 3.

    JSON Syntax / Formatting Slips (12%): Malformed JSON delimiters or trailing commas.JSON 语法/格式错误 (12%):格式错误的 JSON 分隔符或尾随逗号。

This error distribution shows that small-model degradation stems from structured output adherence rather than reasoning capacity, motivating constrained decoding in future runtime iterations.这种错误分布表明,小模型的性能下降源于对结构化输出的遵循,而不是推理能力,这促使在未来的运行时迭代中使用受限解码。

6 Conclusion6 结论

We presented SKILL.state, a runtime architecture that replaces append-only conversational history with explicit, structured execution state. By discarding intermediate reasoning traces after each validated transition, SKILL.state maintains a bounded 𝒪⁡(1)\mathcal{O}(1) prompt footprint and scales linearly 𝒪⁡(T)\mathcal{O}(T) in cumulative tokens. Across controlled diagnostic tasks and public interactive benchmarks, explicit execution state consistently improves task accuracy while substantially reducing prompt growth and token consumption.我们介绍了 SKILL.state,这是一种用显式的、结构化的执行状态取代仅能追加的对话历史的运行时架构。通过在每次经过验证的转换后丢弃中间推理轨迹,SKILL.state 保持有界的 O(1) 提示词占用空间,并在累积 Token 上呈线性 O(T) 扩展。在受控诊断任务和公共交互式基准测试中,显式执行状态始终提高了任务准确性,同时大幅减少了提示词增长和 Token 消耗。

7 Limitations7 局限性

SKILL.state assumes that the execution state can be made a sufficient statistic for future execution: that everything in the past bearing on future actions can be projected into the structured state as soon as it becomes known. Where this holds, discarding intermediate reasoning and conversational history is lossless. However, this assumption fails in three distinct settings: (1) when no fixed schema is known in advance and the relevant state structure must be discovered dynamically during execution; (2) when a correct state update depends on an earlier observation whose relevance was not recognized when first observed, and was therefore never committed to state; and (3) when the task objective is defined over the historical trajectory itself (e.g., auditing, debugging provenance, or explaining past actions), where interaction history is the target output rather than operational overhead.SKILL.state 假设执行状态可以成为未来执行的充分统计量:即过去所有对未来动作有影响的事物,一旦被获知,就可以立即投影到结构化状态中。在这种情况下,丢弃中间推理和对话历史是无损的。然而,该假设在三种不同设置中失效:(1) 当无法预先知道固定模式,且相关状态结构必须在执行过程中动态发现时;(2) 当正确的状态更新依赖于早期的观察结果,而该结果在首次观察时未被识别出相关性,因此从未提交到状态中时;(3) 当任务目标是针对历史轨迹本身定义时(例如审计、调试来源或解释过去动作),此时交互历史是目标输出而非操作开销。

Our current implementation focuses on single-agent procedural execution. While the explicit state abstraction extends naturally to multi-agent systems—where a shared execution state acts as the central coordination substrate instead of exchanging quadratic conversational transcripts—multi-agent environments introduce concurrent writes, requiring deterministic conflict-resolution semantics in the merge operator ⊕\oplus that our single-agent setting does not exercise.我们目前的实现专注于单智能体程序化执行。虽然显式状态抽象自然地扩展到多智能体系统——其中共享执行状态充当中央协调基质,而不是交换二次方的对话记录——但多智能体环境引入了并发写入,需要合并运算符 ⊕ 中具有确定性的冲突解决语义,而我们的单智能体设置未涉及此点。

Finally, SKILL.state relies on the language model to propose valid structured state patches. Because schema ownership and validation reside in the deterministic runtime rather than the model, malformed outputs cannot corrupt persistent state Σt\Sigma_{t}; an invalid patch triggers a rollback-retry cycle. For smaller open-weight models, integrating grammar-constrained decoding can eliminate syntactic formatting errors, allowing the model to focus entirely on semantic state transitions.最后,SKILL.state 依赖语言模型来提出有效的结构化状态补丁。由于模式所有权和验证驻留在确定性运行时而不是模型中,格式错误的输出无法损坏持久状态 Σt;无效补丁会触发回滚重试循环。对于较小的开源权重模型,集成语法约束解码可以消除语法格式错误,使模型能够完全专注于语义状态转换。

References参考文献

  • Badhe et al. (2026) S. Badhe, D. Shah, P. Tiwari, and N. Kathrotia A systematic survey of agent skills: lifecycle, taxonomy, and security. Taxonomy, and Security (July 31, 2026). Cited by: §1, §2.1. Badhe et al. (2026) S. Badhe, D. Shah, P. Tiwari, and N. Kathrotia 智能体技能系统综述:生命周期、分类学与安全。 分类学与安全(2026 年 7 月 31 日)。 引用:§1, §2.1.
  • Badhe and Tiwari (2026) S. Badhe and P. Tiwari Agent skill security: threat models, attacks, defenses, and evaluation. External Links: 2607.13987, Link Cited by: §2.1. Badhe and Tiwari (2026) S. Badhe and P. Tiwari 智能体技能安全:威胁模型、攻击、防御与评估。 外部链接:2607.13987, 链接 引用:§2.1.
  • Chhikara et al. (2025) P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav Mem0: building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413. Cited by: §2.2. Chhikara et al. (2025) P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav Mem0:构建具有可扩展长期记忆的生产就绪 AI 智能体。 arXiv 预印本 arXiv:2504.19413。 引用:§2.2.
  • Heck et al. (2020) M. Heck, C. van Niekerk, N. Lubis, C. Geishauser, H. Lin, M. Moresi, and M. Gašić TripPy: a triple copy strategy for value-independent neural dialog state tracking. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 35–44. Cited by: §2.3. Heck et al. (2020) M. Heck, C. van Niekerk, N. Lubis, C. Geishauser, H. Lin, M. Moresi, and M. Gašić TripPy:一种用于值无关神经对话状态跟踪的三重复制策略。 在第 21 届话语与对话特别兴趣小组年会论文集中, 第 35–44 页。 引用:§2.3.
  • Henderson et al. (2014) M. Henderson, B. Thomson, and J. D. Williams The second dialog state tracking challenge. In Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL), pp. 263–272. Cited by: §2.3. Henderson et al. (2014) M. Henderson, B. Thomson, and J. D. Williams 第二次对话状态跟踪挑战赛。 在第 15 届话语与对话特别兴趣小组(SIGDIAL)年会论文集中, 第 263–272 页。 引用:§2.3.
  • Hosseini-Asl et al. (2020) E. Hosseini-Asl, B. McCann, C. Wu, S. Yavuz, and R. Socher A simple language model for task-oriented dialogue. Advances in Neural Information Processing Systems 33, pp. 20179–20191. Cited by: §2.3. Hosseini-Asl et al. (2020) E. Hosseini-Asl, B. McCann, C. Wu, S. Yavuz, and R. Socher 一种用于任务导向型对话的简单语言模型。 神经信息处理系统进展 33,第 20179–20191 页。 引用:§2.3.
  • Jiang et al. (2023) H. Jiang, Q. Wu, C. Lin, Y. Yang, and L. Qiu LLMLingua: compressing prompts for accelerated inference of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 13358–13376. Cited by: §2.4, item 3. Jiang et al. (2023) H. Jiang, Q. Wu, C. Lin, Y. Yang, and L. Qiu LLMLingua:压缩提示词以加速大型语言模型的推理。 在 2023 年自然语言处理经验方法会议(EMNLP)论文集中, 第 13358–13376 页。 引用:§2.4, 第 3 项。
  • Li et al. (2023) Y. Li, B. Dong, F. Zhang, D. Wang, Y. Xu, X. Chen, and X. Ren Compressing context to enhance inference efficiency of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6342–6353. Cited by: §2.4. Li et al. (2023) Y. Li, B. Dong, F. Zhang, D. Wang, Y. Xu, X. Chen, and X. Ren 压缩上下文以提高大型语言模型的推理效率。 在 2023 年自然语言处理经验方法会议(EMNLP)论文集中, 第 6342–6353 页。 引用:§2.4.
  • Liu et al. (2024) N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang Lost in the middle: how language models use long contexts. Transactions of the Association for Computational Linguistics 12, pp. 157–173. External Links: Link, Document Cited by: §1, §2.4. Liu et al. (2024) N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang 迷失在中间:语言模型如何使用长上下文。 计算语言学协会汇刊 12,第 157–173 页。 外部链接:链接, 文档 引用:§1, §2.4.
  • Mialon et al. (2023) G. Mialon, R. Dessi, M. Lomeli, C. Nalmpantis, R. Pasunuru, R. Raileanu, B. Roziere, T. Schick, J. Dwivedi-Yu, A. Celikyilmaz, E. Grave, Y. LeCun, and T. Scialom Augmented language models: a survey. Transactions on Machine Learning Research. Note: Survey Certification External Links: ISSN 2835-8856, Link Cited by: §1. Mialon et al. (2023) G. Mialon, R. Dessi, M. Lomeli, C. Nalmpantis, R. Pasunuru, R. Raileanu, B. Roziere, T. Schick, J. Dwivedi-Yu, A. Celikyilmaz, E. Grave, Y. LeCun, and T. Scialom 增强型语言模型:综述。 机器学习研究汇刊。 注意:综述认证 外部链接:ISSN 2835-8856, 链接 引用:§1.
  • Packer et al. (2023) C. Packer, V. Fang, S. Patil, K. Lin, S. Wooders, and J. Gonzalez MemGPT: towards llms as operating systems.. Cited by: §1, item 2. Packer et al. (2023) C. Packer, V. Fang, S. Patil, K. Lin, S. Wooders, and J. Gonzalez MemGPT:迈向作为操作系统的 LLM。 引用:§1, 第 2 项。
  • Park et al. (2023) J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein Generative agents: interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST), Cited by: §2.2. Park et al. (2023) J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein 生成式智能体:人类行为的交互式模拟。 在第 36 届 ACM 用户界面软件与技术年度研讨会(UIST)论文集中, 引用:§2.2.
  • Qin et al. (2024) Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, et al. ToolLLM: facilitating large language models to master 16000+ real-world apis. In International Conference on Learning Representations (ICLR), Cited by: §1. Qin et al. (2024) Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, et al. ToolLLM:促进大型语言模型掌握 16000 多种现实世界 API。 在国际学习表示会议(ICLR)上, 引用:§1.
  • Rastogi et al. (2020) A. Rastogi, X. Zang, S. Sunkara, R. Gupta, and P. Khaitan Towards scalable multi-domain conversational agents: the schema-guided dialogue dataset. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, pp. 8689–8696. Cited by: §2.3. Rastogi et al. (2020) A. Rastogi, X. Zang, S. Sunkara, R. Gupta, and P. Khaitan 迈向可扩展的多领域对话智能体:模式引导对话数据集。 在 AAAI 人工智能会议论文集中, 第 34 卷,第 8689–8696 页。 引用:§2.3.
  • Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer:语言模型可以自学使用工具。 在神经信息处理系统进展(NeurIPS)中, 引用:§1.
  • Wang et al. (2023) W. Wang, L. Dong, H. Cheng, X. Liu, X. Yan, J. Gao, and F. Wei Augmenting language models with long-term memory. Advances in Neural Information Processing Systems (NeurIPS). Cited by: §1. Wang et al. (2023) W. Wang, L. Dong, H. Cheng, X. Liu, X. Yan, J. Gao, and F. Wei 用长期记忆增强语言模型。 神经信息处理系统进展(NeurIPS)。 引用:§1.
  • Williams et al. (2013) J. Williams, A. Raux, D. Ramachandran, and A. Black The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference, pp. 404–413. Cited by: §2.3. Williams et al. (2013) J. Williams, A. Raux, D. Ramachandran, and A. Black 对话状态跟踪挑战赛。 在 SIGDIAL 2013 会议论文集中, 第 404–413 页。 引用:§2.3.
  • Wu et al. (2019) C. Wu, A. Madotto, E. Hosseini-Asl, C. Xiong, R. Socher, and P. Fung Transferable multi-domain state generator for task-oriented dialogue systems. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 808–819. Cited by: §2.3. Wu et al. (2019) C. Wu, A. Madotto, E. Hosseini-Asl, C. Xiong, R. Socher, and P. Fung 用于任务导向型对话系统的可迁移多领域状态生成器。 在第 57 届计算语言学协会年会论文集中, 第 808–819 页。 引用:§2.3.
  • Wu et al. (2023) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. AutoGen: enabling next-gen llm applications via multi-agent conversation. arXiv preprint arXiv:2308.08155. Cited by: §1. Wu et al. (2023) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. AutoGen:通过多智能体对话实现下一代 LLM 应用。 arXiv 预印本 arXiv:2308.08155。 引用:§1.
  • Xiao et al. (2024a) G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis Efficient streaming language models with attention sinks. In International Conference on Learning Representations (ICLR), Cited by: §1. Xiao et al. (2024a) G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis 具有注意力汇聚的高效流式语言模型。 在国际学习表示会议(ICLR)上, 引用:§1.
  • Xiao et al. (2024b) G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis Efficient streaming language models with attention sinks. In International Conference on Learning Representations, Vol. 2024, pp. 21875–21895. Cited by: §2.4. Xiao et al. (2024b) G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis 具有注意力汇聚的高效流式语言模型。 在国际学习表示会议上, 第 2024 卷,第 21875–21895 页。 引用:§2.4.
  • Yang et al. (2023) J. Yang, A. Prabhakar, K. Narasimhan, and S. Yao InterCode: standardizing and benchmarking interactive coding with execution feedback. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, 1st item. Yang et al. (2023) J. Yang, A. Prabhakar, K. Narasimhan, and S. Yao InterCode:通过执行反馈标准化和基准测试交互式编码。 在神经信息处理系统进展(NeurIPS)中, 引用:§1, 第 1 项。
  • Yao et al. (2024) S. Yao, N. Shinn, J. Zhao, Q. Wu, and K. Narasimhan τ\tau-Bench: a benchmark for tool-agent-user interaction in real-world domains. arXiv preprint arXiv:2406.12045. Cited by: §1, 2nd item. Yao et al. (2024) S. Yao, N. Shinn, J. Zhao, Q. Wu, and K. Narasimhan τ-Bench:一个用于真实领域中工具-智能体-用户交互的基准。 arXiv 预印本 arXiv:2406.12045。 引用:§1, 第 2 项。
  • Yao et al. (2022) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao React: synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Cited by: §1, §1, item 1. Yao et al. (2022) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao React:在语言模型中协同推理和行动。 arXiv 预印本 arXiv:2210.03629。 引用:§1, §1, 第 1 项。
  • Zhang et al. (2024) X. Zhang, Y. Chen, S. Hu, Z. Xu, J. Chen, M. Hao, X. Han, Z. Thai, S. Wang, Z. Liu, and M. Sun ∞\inftyBench: Extending long context evaluation beyond 100K tokens. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, pp. 15262–15277. External Links: Link, Document Cited by: §2.4. Zhang et al. (2024) X. Zhang, Y. Chen, S. Hu, Z. Xu, J. Chen, M. Hao, X. Han, Z. Thai, S. Wang, Z. Liu, and M. Sun ∞Bench:将长上下文评估扩展到 100K Token 之外。 在第 62 届计算语言学协会年会论文集(第 1 卷:长论文)中,L. Ku, A. Martins, 和 V. Srikumar (编), 泰国曼谷,第 15262–15277 页。 外部链接:链接, 文档 引用:§2.4.
  • Zhong et al. (2023) W. Zhong, L. Guo, Q. Gao, H. Y. Wang, and Y. Lin MemoryBank: enhancing large language models with long-term memory. arXiv preprint arXiv:2305.10250. Cited by: §1. Zhong et al. (2023) W. Zhong, L. Guo, Q. Gao, H. Y. Wang, and Y. Lin MemoryBank:用长期记忆增强大型语言模型。 arXiv 预印本 arXiv:2305.10250。 引用:§1.
  • Zhong et al. (2024) W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang Memorybank: enhancing large language models with long-term memory. In Proceedings of the AAAI conference on artificial intelligence, Vol. 38, pp. 19724–19731. Cited by: §2.2. Zhong et al. (2024) W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang Memorybank:用长期记忆增强大型语言模型。 在 AAAI 人工智能会议论文集中, 第 38 卷,第 19724–19731 页。 引用:§2.2.

Appendix A. Runtime Prompts附录 A. 运行时提示词

This appendix provides the exact system prompts used by the four evaluated runtimes. To ensure reproducibility, all prompts are presented exactly as they were dynamically constructed and formatted in the benchmark execution loop.本附录提供了四种被评估运行时所使用的确切系统提示词。为确保可复现性,所有提示词均按其在基准测试执行循环中动态构建和格式化的方式呈现。

A.1 Prompt Runtime (ReAct-style)A.1 提示词运行时(ReAct 风格)

Instructions指令:
{skill技能.instructions指令}
History历史记录:
Observation观察: {history历史记录[0].[0]。observation观察}
Reasoning推理 & Action行动: {history[0].response响应}
[...[... Appends追加 all所有 previous之前的 observations观察 and和 actions行动 ...]...]
Latest最新 Observation: {observation}
Generate生成 your你的 next下一个 reasoning推理 and action行动 (format格式 ’Action: <cmd命令>’):>’):

A.2 Memory-Augmented RuntimeA.2 记忆增强运行时

Instructions:
{skill.instructions}
Summarized已总结 History:
{summary_string_of_past_steps过去步骤的摘要字符串}
Recent最近 History:
Observation: {recent_observations最近的观察[0]}[0]}
Response响应: {recent_responses最近的响应[0]}
[... Appends the最 3 most近 recent的 turns轮次 ...]
Latest Observation: {observation}
Generate your next reasoning and action (format ’Action: <cmd>’):

A.3 Stateful Runtime (LangGraph-style)A.3 有状态运行时(LangGraph 风格)

Instructions:
{skill.instructions}
Current当前 State状态:
{jsonjson.dumps转储(state状态, indent缩进=2)}=2)}
History:
Observation: {history[0].observation}
Response: {history[0].response}
[... Appends all previous observations and actions ...]
Latest Observation: {observation}
Update更新 the state if necessary必要, provide提供 reasoning, and output输出 ’Action: <cmd>’.>’。
To update更新 state, use使用 the format: StateUpdate状态更新: {"key键": "value值"}

A.4 SKILL.state RuntimeA.4 SKILL.state 运行时

Instructions:
{skill.instructions}
Skill技能 Execution执行 State:
‘‘‘‘‘‘json
{json.dumps(state, separators分隔符=(’,’,=(’,’, ’:’))}’:’))}
Latest Observation: {observation}
Provide提供 your response with带有:
1. Step步骤-by-step步骤 reasoning (will将 be discarded被丢弃 after在执行后 execution执行)
2. A JSON block块 fenced围栏 with json ...... containing包含 both两者 your State Patch补丁 and your Action. The该 JSON block MUST have具有 exactly确切的 these这两个 two键 keys键: { "state_patch状态补丁": { <dict字典: your state updates更新, set设置 keys to null空 to delete删除> }, "action": "<string字符串: the exact确切的 command命令 you你 want想要 to execute执行>" }

Note: The skill.instructions placeholder dynamically injects the task-specific system prompt (e.g., the agent’s persona, the available action space, and the environment rules). This ensures that across all runtime evaluations, the agent receives the exact same baseline instructions, isolating context management as the only independent variable.注意:skill.instructions 占位符动态注入任务特定的系统提示词(例如,代理的角色、可用的行动空间和环境规则)。这确保了在所有运行时评估中,代理都能接收到完全相同的基准指令,从而将上下文管理作为唯一的独立变量进行隔离。

Appendix B. SkillExecBench Implementation Details附录 B. SkillExecBench 实现细节

To maintain focus on the core experimental findings in Section 5, we provide the full implementation details of the SkillExecBench environments, task generation logic, and episode trajectories in this appendix.为了保持对第 5 节核心实验发现的关注,我们在本附录中提供了 SkillExecBench 环境、任务生成逻辑和片段轨迹的完整实现细节。

B.1 Environment DesignB.1 环境设计

Environment 1: Warehouse Management环境 1:仓库管理

  • •

    State Representation: A discrete inventory mapping of 500 independent shelves (e.g., shelf_0 through shelf_499), where each shelf holds exactly one item string identifier or is null.状态表示:500 个独立货架的离散库存映射(例如,shelf_0 到 shelf_499),其中每个货架恰好持有一个物品字符串标识符或为空。

  • •

    Action Space:行动空间:

    • –

      Store <item_id> <empty_shelf_id>Store <item_id> <empty_shelf_id>

    • –

      Ship <item_id> <shelf_id>Ship <item_id> <shelf_id>

    • –

      Move <item_id> <old_shelf_id> <new_shelf_id>Move <item_id> <old_shelf_id> <new_shelf_id>

    • –

      WaitWait

  • •

    Observation Format: Textual alerts triggered by system events, including: Shipment arrived containing [item], Customer ordered [item], and Maintenance required on [shelf].观察格式:由系统事件触发的文本警报,包括:货物已到达,包含 [item];客户订购了 [item];以及 [shelf] 需要维护。

  • •

    Transition Rules: If an agent calls Store, the environment validates the shelf is empty before placing the item. If Ship is called, the item is destroyed. Invalid actions (e.g., storing an item on an occupied shelf) return a local error observation and reject the state transition.转换规则:如果代理调用 Store,环境会在放置物品前验证货架是否为空。如果调用 Ship,则物品被销毁。无效的行动(例如,将物品存储在已占用的货架上)会返回本地错误观察并拒绝状态转换。

  • •

    Success Criterion: The ratio of successfully executed valid actions matching the ground-truth deterministic simulation (Score = Successful Actions / Total Actionable Events).成功标准:匹配地面实况确定性模拟的成功执行有效行动的比率(分数 = 成功行动 / 总可行动事件)。

Environment 2: Software Repository环境 2:软件仓库

  • •

    State Representation: A simulated Git repository tracking branch histories, file contents, active Pull Requests (PRs), and Continuous Integration (CI) test statuses.状态表示:一个模拟的 Git 仓库,跟踪分支历史、文件内容、活动的拉取请求 (PR) 以及持续集成 (CI) 测试状态。

  • •

    Action Space: Commit(branch, file), CreatePR(branch), Merge(pr_id), FixCI(branch), Wait.行动空间:Commit(branch, file), CreatePR(branch), Merge(pr_id), FixCI(branch), Wait。

  • •

    Observation Format: CI/CD webhook notifications (e.g., CI Pipeline Failed for PR #3), code review comments, and issue assignments.观察格式:CI/CD Webhook 通知(例如,PR #3 的 CI 流水线失败)、代码审查评论和问题分配。

  • •

    Transition Rules: Pushing a commit triggers a background CI evaluation transition. Merging a PR successfully transitions the master branch state and deletes the feature branch.转换规则:推送提交会触发后台 CI 评估转换。合并 PR 会成功转换主分支状态并删除功能分支。

  • •

    Success Criterion: The percentage of correctly resolved feature requests merged into master without breaking CI checks.成功标准:成功合并到主分支且未破坏 CI 检查的功能请求百分比。

B.2 Task GenerationB.2 任务生成

Tasks are procedurally generated using a deterministic random number generator. By seeding the generator, we guarantee that all evaluated baseline runtimes experience the exact same sequence of events, ensuring fair comparisons. The horizon length dictates the number of sequential events the agent must handle.任务使用确定性随机数生成器按程序生成。通过对生成器设置种子,我们保证所有被评估的基准运行时都会经历完全相同的事件序列,从而确保公平比较。视野长度决定了代理必须处理的顺序事件数量。

Algorithm 2 Warehouse Task Generation Pseudo-code算法 2 仓库任务生成伪代码
1: s​e​e​d←42seed\leftarrow 42
2: r​n​g←Random​(s​e​e​d)rng\leftarrow\text{Random}(seed)
3: gt_shelves←{shelf_0: Null, …, shelf_499: Null}gt\_shelves\leftarrow\{\text{shelf\_0: Null, ..., shelf\_499: Null}\}
4: e​v​e​n​t​s←[]events\leftarrow[]
5: for i=1i=1 to H​o​r​i​z​o​nHorizon do
6:   a​v​a​i​l​a​b​l​e←FindEmptyShelves​(g​t​_​s​h​e​l​v​e​s)available\leftarrow\text{FindEmptyShelves}(gt\_shelves)
7:   o​c​c​u​p​i​e​d←FindOccupiedShelves​(g​t​_​s​h​e​l​v​e​s)occupied\leftarrow\text{FindOccupiedShelves}(gt\_shelves)
8:   p​o​s​s​i​b​l​e​_​e​v​e​n​t​s←[Receive]possible\_events\leftarrow[\text{Receive}]
9:   if o​c​c​u​p​i​e​d​ is not emptyoccupied\text{ is not empty} then那么
10:10:    p​o​s​s​i​b​l​e​_​e​v​e​n​t​s​.append​(Order,Maintenance)possible\_events\text{.append}(\text{Order},\text{Maintenance})
11:11:   end if
12:12:   e​v​e​n​t​_​t​y​p​e←r​n​g​.choice​(p​o​s​s​i​b​l​e​_​e​v​e​n​t​s)event\_type\leftarrow rng\text{.choice}(possible\_events)
13:13:   e​v​e​n​t​s​.append​(ConstructObservation​(e​v​e​n​t​_​t​y​p​e))events\text{.append}(\text{ConstructObservation}(event\_type))
14:14:   UpdateGroundTruth​(g​t​_​s​h​e​l​v​e​s,e​v​e​n​t​_​t​y​p​e)\text{UpdateGroundTruth}(gt\_shelves,event\_type)
15:15: end for
16:16: return返回 e​v​e​n​t​sevents

B.3 Example Episode TrajectoryB.3 示例片段轨迹

To illustrate the execution flow, below is a concrete, single-turn trajectory for the SKILL.state runtime operating in the Warehouse environment.为了说明执行流程,下面是在仓库环境中运行的 SKILL.state 运行时的具体单轮轨迹。

1. Observation Received (Environment →\rightarrow LLM)1. 收到观察(环境 → LLM)

Latest Observation:
Customer客户 ordered订购了 item_12item_12.

2. LLM Processing (LLM Internal)2. LLM 处理(LLM 内部)

Reasoning:
The customer客户 ordered item_12. I need需要 to ship发货 it.
I will check检查 my Skill Execution State to locate定位 it.
Looking正在查看 at the state schema架构, item_12 is located定位到 on shelf_42shelf_42.
I must必须 generate生成 a ShipShip action and update my state to remove移除 it from从 shelf_42.

3. State Update and Action Generation (LLM →\rightarrow Runtime)3. 状态更新和行动生成(LLM → 运行时)

{
"state_patch": {
"inventory库存": {
"shelf_42": null
}
},
"action": "Ship item_12 shelf_42"
}

4. Next Observation (Environment →\rightarrow LLM)4. 下一个观察(环境 → LLM)

Latest Observation:
Success成功: Shipped已发货 item_12 from shelf_42.

Appendix C. Noise Construction (Experiment 2)附录 C. 噪声构建(实验 2)

In Experiment 2, we evaluate the runtimes’ resilience to dense, irrelevant contextual noise. Real-world systems rarely provide clean, perfectly isolated observation spaces; agents must constantly filter out background telemetry, sensor logs, and system chatter to execute their instructions.在实验 2 中,我们评估了运行时对密集、无关上下文噪声的韧性。现实世界的系统很少提供干净、完全隔离的观察空间;代理必须不断过滤掉后台遥测、传感器日志和系统杂音才能执行指令。

To isolate the problem of Attention Drag, the experiments in this paper exclusively focus on Condition 1: Irrelevant Context.为了隔离“注意力拖累”(Attention Drag)问题,本文中的实验专门关注条件 1:无关上下文。

C.1 Noise PropertiesC.1 噪声属性

For Condition 1 evaluations across both environments, the injected noise strings are defined by three strict properties:对于两个环境中条件 1 的评估,注入的噪声字符串由三个严格属性定义:

  1. 1.

    Randomly Generated: Values such as battery percentages, temperatures, server IDs, and CPU loads are sampled uniformly at random during each execution step.随机生成:电池百分比、温度、服务器 ID 和 CPU 负载等值在每个执行步骤中均从均匀分布中随机采样。

  2. 2.

    Strictly Irrelevant: The semantic meaning of the noise has absolutely no bearing on the agent’s primary task (e.g., fulfilling warehouse orders or fixing CI pipelines).严格无关:噪声的语义含义与代理的主要任务(例如,履行仓库订单或修复 CI 流水线)完全无关。

  3. 3.

    Non-State-Altering: The noise events never actually change the underlying ground-truth world state. They are purely observational distractors appended to the environment’s response payload under a --- BACKGROUND TELEMETRY --- header.非状态改变:噪声事件绝不会真正改变底层的地面实况世界状态。它们纯粹是附加在环境响应负载 --- BACKGROUND TELEMETRY --- 标题下的观察干扰。

C.2 Environment 1 (Warehouse) DistractorsC.2 环境 1(仓库)干扰项

To simulate a realistic noisy warehouse, the generator randomly selects from the following categories to inject irrelevant strings into the agent’s observation space:为了模拟逼真的嘈杂仓库,生成器会随机选择以下类别,将无关字符串注入代理的观察空间:

1. Robot Telemetry Logs1. 机器人遥测日志

Simulates continuous pinging from automated warehouse robots navigating the floor.模拟在地面上导航的自动仓库机器人的持续 Ping 信号。

Battery电池: 85%,85%, Temperature温度: 45C, CPU Load负载: 72%,72%,
Speed速度: 1.21.2 m/s, Nav导航 Confidence置信度: 95.4%95.4%

2. Environmental Sensor Logs2. 环境传感器日志

Simulates passive HVAC and ambient sensor readings.模拟被动式 HVAC 和环境传感器读数。

[Sensor传感器] Humidity湿度: 45%,45%, Temp温度: 22.322.3C, Light光照: 310310 luxlux, CO2: 450450 ppmppm

3. Camera OCR / Vision Logs3. 摄像头 OCR / 视觉日志

Simulates background security camera or computer-vision object detection events.模拟后台安全摄像头或计算机视觉对象检测事件。

[Camera摄像头 OCR] Forklift叉车 parked已停放.
[Camera OCR] Worker工人 entered进入 Zone区域 A.
[Camera OCR] Safety安全 Vest背心 Detected检测到.

C.3 Environment 2 (Software Repository) DistractorsC.3 环境 2(软件仓库)干扰项

To simulate a noisy, enterprise-scale software engineering environment, the generator continuously injects irrelevant cloud infrastructure syslog telemetry into the agent’s terminal observations.为了模拟嘈杂的企业级软件工程环境,生成器不断向代理的终端观察中注入无关的云基础设施系统日志遥测。

Syslog Telemetry系统日志遥测

Simulates passive health-checks and CPU load warnings from disconnected remote servers running in the background.模拟后台运行的已断开连接的远程服务器的被动健康检查和 CPU 负载警告。

[Syslog系统日志] Server服务器-42-42 CPU load负载: 88%,88%, RAM usage使用率: 71%71%
[Syslog] Server-17-17 CPU load: 12%,12%, RAM usage: 45%45%
[Syslog] Server-91-91 CPU load: 99%,99%, RAM usage: 89%89%

Example Corrupted Observation (Software, 3 Events):示例损坏的观察(软件,3 个事件):

Latest Observation:
CI Pipeline流水线 Failed失败 for PR #3.#3。 LinterLinter error错误 on line行 42.42。
------ BACKGROUND TELEMETRY ---
[Syslog] Server-42 CPU load: 88%, RAM usage: 71%
[Syslog] Server-17 CPU load: 12%, RAM usage: 45%
[Syslog] Server-91 CPU load: 99%, RAM usage: 89%

Appendix D. Additional Results附录 D. 补充结果

Table 6: Software Repository Long-Horizon Execution Scaling using using Gemini-3-Flash. Baseline runtimes suffer catastrophic O⁡(N2)O(N^{2}) context collapse, whereas SKILL.state maintains an O⁡(1)O(1) prompt footprint.表 6:使用 Gemini-3-Flash 的软件仓库长视野执行扩展。基准运行时遭受灾难性的 O(N²) 上下文崩溃,而 SKILL.state 保持 O(1) 的提示词占用空间。
Horizon Runtime Score Avg Prompt Size Total Tokens Consumed
10 Prompt 0.89 ±\pm0.11 3,411 ±\pm197 11,670 ±\pm841
Memory 0.93 ±\pm0.09 4,379 ±\pm234 15,732 ±\pm562
Stateful 1.00 ±\pm0.00 4,200 ±\pm321 14,120 ±\pm318
SKILL.state 1.00 ±\pm0.00 2,298 ±\pm134 7,608 ±\pm149
25 Prompt 0.84 ±\pm0.05 11,754 ±\pm608 111,970 ±\pm3,314
Memory 0.89 ±\pm0.07 9,399 ±\pm317 94,629 ±\pm2,839
Stateful 0.94 ±\pm0.03 14,016 ±\pm586 128,702 ±\pm3,863
SKILL.state 0.88 ±\pm0.08 2,545 ±\pm556 21,920 ±\pm431
50 Prompt 0.71 ±\pm0.14 23,136 ±\pm911 462,118 ±\pm13,764
Memory 0.65 ±\pm0.12 35,550 ±\pm2,412 688,182 ±\pm23,539
Stateful 0.74 ±\pm0.08 31,166 ±\pm3,231 577,027 ±\pm27,293
SKILL.state 0.86 ±\pm0.04 2,545 ±\pm63 45,100 ±\pm894
100 Prompt 0.53 ±\pm0.16 46,270 ±\pm1,847 1,848,500 ±\pm55,391
Memory 0.57 ±\pm0.05 71,100 ±\pm5,836 2,752,700 ±\pm82,467
Stateful 0.63 ±\pm0.10 62,330 ±\pm2,488 2,308,000 ±\pm35,183
SKILL.state 0.78 ±\pm0.08 2,545 ±\pm471 90,200 ±\pm2,792
Table 7: Gemma-4-31b-it Warehouse Scaling.表 7:Gemma-4-31b-it 仓库扩展。
Horizon Runtime Score ±\pm SD Avg Prompt ±\pm SD Total Tokens ±\pm SD
10 Steps Prompt 0.90 ±\pm 3.1% 3,145 ±\pm 242 9,092 ±\pm 1,231
Memory 0.85 ±\pm 4.2% 2,611 ±\pm 138 7,330 ±\pm 838
Stateful 0.90 ±\pm 2.8% 3,191 ±\pm 149 9,144 ±\pm 518
SKILL.state 0.98 ±\pm 1.5% 2,116 ±\pm 212 6,814 ±\pm 875
25 Steps Prompt 0.64 ±\pm 5.4% 5,720 ±\pm 182 41,697 ±\pm 1,610
Memory 0.72 ±\pm 4.8% 3,990 ±\pm 362 28,933 ±\pm 412
Stateful 0.76 ±\pm 4.1% 5,714 ±\pm 282 39,314 ±\pm 585
SKILL.state 0.84 ±\pm 3.6% 2,080 ±\pm 114 16,302 ±\pm 2,190
50 Steps Prompt 0.31 ±\pm 6.2% 10,809 ±\pm 352 151,845 ±\pm 2,150
Memory 0.41 ±\pm 5.8% 7,217 ±\pm 316 114,113 ±\pm 1,620
Stateful 0.55 ±\pm 5.1% 11,083 ±\pm 262 155,164 ±\pm 2,210
SKILL.state 0.68 ±\pm 3.9% 2,113 ±\pm 176 33,762 ±\pm 1,385
100 Steps Prompt 0.21 ±\pm 4.7% 27,686 ±\pm 412 923,164 ±\pm 12,400
Memory 0.24 ±\pm 4.2% 18,537 ±\pm 229 701,954 ±\pm 9,850
Stateful 0.42 ±\pm 4.5% 20,210 ±\pm 822 557,968 ±\pm 8,100
SKILL.state 0.42 ±\pm 4.1% 2,105 ±\pm 216 65,480 ±\pm 1,258
Table 8: Qwen 3-8b-it Warehouse Scaling.表 8:Qwen 3-8b-it 仓库扩展。
Horizon Runtime Score ±\pm SD Avg Prompt ±\pm SD Total Tokens ±\pm SD
10 Steps Prompt 0.84 ±\pm 3.8% 3,150 ±\pm 245 9,150 ±\pm 1,250
Memory 0.80 ±\pm 4.5% 2,640 ±\pm 145 7,420 ±\pm 860
Stateful 0.84 ±\pm 3.4% 3,210 ±\pm 155 9,210 ±\pm 540
SKILL.state 0.94 ±\pm 2.1% 2,120 ±\pm 215 6,920 ±\pm 890
25 Steps Prompt 0.54 ±\pm 6.1% 5,790 ±\pm 195 42,450 ±\pm 1,680
Memory 0.62 ±\pm 5.4% 4,050 ±\pm 375 29,640 ±\pm 440
Stateful 0.66 ±\pm 4.8% 5,780 ±\pm 295 39,950 ±\pm 610
SKILL.state 0.76 ±\pm 4.2% 2,088 ±\pm 120 16,680 ±\pm 2,240
50 Steps Prompt 0.24 ±\pm 6.5% 10,950 ±\pm 365 154,200 ±\pm 2,280
Memory 0.33 ±\pm 6.1% 7,320 ±\pm 330 116,400 ±\pm 1,710
Stateful 0.44 ±\pm 5.7% 11,210 ±\pm 280 158,300 ±\pm 2,340
SKILL.state 0.58 ±\pm 4.5% 2,118 ±\pm 185 34,510 ±\pm 1,420
100 Steps Prompt 0.15 ±\pm 5.1% 28,150 ±\pm 430 941,500 ±\pm 12,800
Memory 0.18 ±\pm 4.6% 18,840 ±\pm 245 718,200 ±\pm 10,200
Stateful 0.31 ±\pm 4.9% 20,580 ±\pm 850 569,400 ±\pm 8,450
SKILL.state 0.34 ±\pm 4.6% 2,110 ±\pm 220 66,850 ±\pm 1,310
Table 9: Software Repository Experiment 2 i.e. Noise Robustness. Evaluation of runtime resilience to irrelevant syslog telemetry (Condition 1) during a 50-step horizon using Gemini-3-Flash.表 9:软件仓库实验 2,即噪声稳健性。使用 Gemini-3-Flash 在 50 步视野内评估运行时对无关系统日志遥测(条件 1)的韧性。
Noise Level Runtime Score
0 Events Prompt 0.76
(Baseline) Memory 0.85
Stateful 0.88
SKILL.state 0.90
5 Events Prompt 0.62
(Low) Memory 0.85
Stateful 0.86
SKILL.state 0.88
20 Events Prompt 0.48
(Medium) Memory 0.83
Stateful 0.85
SKILL.state 0.86
50 Events Prompt 0.11
(High) Memory 0.74
Stateful 0.78
SKILL.state 0.80

.1 Software Repository State Recovery Experiment 3 using Gemini-3-Flash..1 使用 Gemini-3-Flash 的软件仓库状态恢复实验 3。

Table 10: Software Repository State Recovery (Env 2, Exp 3). Comparison of hallucination lag (recovery steps) when the repository state is altered via unstructured alerts.表 10:软件仓库状态恢复(环境 2,实验 3)。当仓库状态通过非结构化警报更改时,幻觉滞后(恢复步骤)的比较。
Scenario Runtime Success Recovery Steps
A: Force Push Prompt Yes 12
Memory Yes 8
Stateful Yes 10
SKILL.state Yes 0
B: Flaky CI Test Prompt Yes 14
Memory Yes 9
Stateful Yes 11
SKILL.state Yes 0
C: PR Closed All Runtimes No N/A

.2 Extended Budget-Matched Scaling across Horizons.2 跨视野的扩展预算匹配扩展

Table 11 presents the full multi-horizon scaling results for the budget-matched control configurations on the Warehouse environment (T∈{10,25,50,100}T\in\{10,25,50,100\}, Gemini-3-Flash), evaluated across 5 environmental seeds. While all budget-matched runtimes perform comparably at short horizons (T=10T=10), sliding-window truncation and statistical perplexity compression (LLMLingua) experience catastrophic accuracy collapse as horizon length increases, whereas SKILL.state maintains robust performance across all horizons.表 11 展示了仓库环境中预算匹配控制配置的完整多视野扩展结果(T∈{10,25,50,100},Gemini-3-Flash),在 5 个环境种子下进行评估。虽然所有预算匹配的运行时在短视野(T=10)下表现相当,但滑动窗口截断和统计困惑度压缩(LLMLingua)随着视野长度的增加经历了灾难性的准确度崩溃,而 SKILL.state 在所有视野中均保持稳健表现。

Table 11: Extended Budget-Matched Scaling on Warehouse across Horizons (Gemini-3-Flash, Target Budget ∼\sim1,800 tokens, Score ±\pm SD across 5 seeds).表 11:仓库跨视野的扩展预算匹配扩展(Gemini-3-Flash,目标预算 ∼1,800 token,5 个种子的分数 ± 标准差)。
Horizon (TT) SKILL.state Summary-capped Truncated (Window) ReAct + LLMLingua ReAct (Full)
10 1.00 ±\pm0.00 0.92 ±\pm0.03 0.90 ±\pm0.04 0.88 ±\pm0.04 0.90 ±\pm0.02
25 1.00 ±\pm0.00 0.76 ±\pm0.04 0.62 ±\pm0.05 0.60 ±\pm0.05 0.92 ±\pm0.02
50 0.96 ±\pm0.01 0.64 ±\pm0.05 0.35 ±\pm0.06 0.38 ±\pm0.06 0.88 ±\pm0.04
100 0.94 ±\pm0.01 0.52 ±\pm0.05 0.18 ±\pm0.05 0.22 ±\pm0.05 0.84 ±\pm0.07