SkillWeaver: Web Agents can Self-Improve by Discovering and Honing SkillsSkillWeaver：Web 智能体通过发现和磨练技能实现自我提升

Introduction介绍

Web automation has long been a goal for artificial intelligence researchers, but creating agents that can reliably navigate and perform tasks across diverse websites remains challenging. Most current approaches struggle with generalization and require extensive human supervision. The paper "SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills" introduces a novel framework that enables web agents to autonomously discover, learn, and refine skills for interacting with websites.Web 自动化长期以来一直是人工智能研究的目标，但创建能够可靠地在不同网站间导航并执行任务的智能体仍然充满挑战。目前大多数方法在泛化能力上存在困难，且需要大量的人工监督。论文《SkillWeaver：Web 智能体通过发现和磨练技能实现自我提升》引入了一种新颖的框架，使 Web 智能体能够自主发现、学习和精炼与网站交互的技能。

SkillWeaver addresses a fundamental question: can AI agents independently discover useful web skills and progressively improve without constant human intervention? This approach mirrors how humans learn to navigate the web—by discovering functionality, practicing interactions, and building a repertoire of reusable skills. The framework allows agents to build an expanding library of structured APIs that encapsulate procedural knowledge, making web interactions more efficient and reliable.SkillWeaver 解决了一个根本性问题：AI 智能体能否在没有持续人工干预的情况下，独立发现有用的 Web 技能并逐步提升？这种方法模拟了人类学习浏览 Web 的方式——通过发现功能、练习交互并构建可重用技能库。该框架允许智能体构建一个不断扩展的结构化 API 库，将程序性知识封装起来，从而使 Web 交互更加高效和可靠。

How SkillWeaver WorksSkillWeaver 的工作原理

SkillWeaver's core innovation lies in its ability to transform successful web interactions into reusable skills encoded as Python functions with Playwright (a web automation library) code. Unlike previous approaches that implicitly store skills through trajectories or require human-written workflows, SkillWeaver autonomously creates explicit, structured APIs that can be shared, debugged, and composed to tackle increasingly complex tasks.SkillWeaver 的核心创新在于它能够将成功的 Web 交互转化为可重用的技能，并编码为带有 Playwright（一种 Web 自动化库）代码的 Python 函数。与以往通过轨迹隐式存储技能或需要人工编写工作流的方法不同，SkillWeaver 自主创建明确、结构化的 API，这些 API 可以被共享、调试和组合，以处理日益复杂的任务。

The framework operates in a continuous learning loop where:该框架在一个持续学习循环中运行：

1. The agent explores websites to discover potential skills智能体探索网站以发现潜在技能
2. It practices these skills and encodes successful attempts as structured APIs它练习这些技能并将成功的尝试编码为结构化 API
3. It tests and refines these APIs to improve reliability它测试并精炼这些 API 以提高可靠性
4. It builds a growing skill library that enhances performance on various web tasks它构建一个不断增长的技能库，从而增强在各种 Web 任务上的表现

This approach fundamentally differs from traditional web agents by focusing on skill discovery and abstraction rather than task-specific training.这种方法通过专注于技能发现和抽象，而不是特定任务的训练，与传统的 Web 智能体有着本质区别。

The Three-Stage Pipeline三阶段流水线

SkillWeaver's workflow consists of three distinct stages:SkillWeaver 的工作流程由三个不同的阶段组成：

Stage I: Skill Proposal第一阶段：技能提议

In this stage, large language models (LLMs) serve as an automatic curriculum generator. When presented with website observations (screenshots, URLs, accessibility trees), the LLM proposes potential skills that could be useful on that website. For example, on a pharmaceutical website, it might propose skills like "identify pills using a pill identifier" or "check side effects for a drug."在此阶段，大语言模型 (LLM) 充当自动课程生成器。当面对网站观察结果（截图、URL、可访问性树）时，LLM 会提出在该网站上可能有用的潜在技能。例如，在医药网站上，它可能会提出“使用药品识别器识别药丸”或“检查药物副作用”等技能。

The skill proposals follow a curriculum progression:技能提议遵循课程进阶逻辑：

• Procedural Tasks: Basic interactions like filling forms or clicking buttons程序性任务：填写表单或点击按钮等基本交互
• Navigational Tasks: Finding and reaching specific pages within a website导航任务：在网站内查找并到达特定页面
• Information-Seeking Tasks: Extracting specific information from web content信息获取任务：从网页内容中提取特定信息

This approach ensures a diverse set of skills that range from simple to complex, providing comprehensive coverage of website functionality.这种方法确保了技能的多样性，涵盖了从简单到复杂的各种需求，从而全面覆盖网站功能。

Stage II: Skill Synthesis第二阶段：技能合成

Once skills are proposed, the agent attempts to execute them using a base agent architecture. This practice phase involves:一旦提出技能，智能体就会尝试使用基础智能体架构来执行它们。此练习阶段包括：

1. The base agent trying to complete the proposed skill基础智能体尝试完成所提议的技能
2. A reward model (implemented using an LLM) evaluating success based on the task description, action trajectory, and environmental feedback奖励模型（使用 LLM 实现）根据任务描述、动作轨迹和环境反馈来评估成功与否
3. For successful attempts, an API synthesis process that generates a structured Python function with:
   • A function signature with parameters带有参数的函数签名
   • A docstring explaining the function's purpose解释函数用途的文档字符串 (docstring)
   • Executable Playwright code that performs the web interactions执行 Web 交互的可运行 Playwright 代码

This synthesis process transforms successful trajectories into reusable code that can be called by other agents, creating an abstraction layer that hides the complexity of web interactions.此合成过程将成功的轨迹转化为可被其他智能体调用的可重用代码，创建了一个隐藏 Web 交互复杂性的抽象层。

Stage III: Skill Honing第三阶段：技能磨练

The final stage involves testing and debugging the synthesized APIs to ensure they work reliably. The process includes:最后阶段涉及测试和调试合成的 API，以确保它们能可靠运行。该过程包括：

1. Generating appropriate test cases with diverse parameter values生成具有不同参数值的适当测试用例
2. Executing the APIs with these test cases使用这些测试用例执行 API
3. Analyzing failures and applying fixes分析故障并应用修复程序
4. Updating the API documentation with usage logs and known issues使用使用日志和已知问题更新 API 文档

This debugging process is crucial for improving API robustness and ensuring they work across different instances of similar tasks. The image shows a debugging example where the agent identifies and fixes a color parameter issue in a pill identification API.此调试过程对于提高 API 的稳健性并确保它们在类似任务的不同实例中正常工作至关重要。图像展示了一个调试示例，其中智能体识别并修复了药丸识别 API 中的颜色参数问题。

Skill API Structure技能 API 结构

The synthesized APIs follow a consistent structure designed for reusability and clarity:合成的 API 遵循一致的结构，旨在实现可重用性和清晰度：

async def identify_pill(page, imprint, color=None, shape=None):
    """
    Automates the process of identifying a pill using the Pill Identifier 
    feature on Drugs.com.
    
    Parameters:
    - page: The Playwright page object.
    - imprint: The imprint on the pill to be identified.
    - color: (Optional) The color of the pill.
    - shape: (Optional) The shape of the pill.
    
    This function navigates to the Pill Identifier page, agrees to the terms,
    inputs the pill's characteristics, and submits the information for identification.
    """
    # Implementation with Playwright code
    # ...

This structure makes the APIs:这种结构使 API 具备以下特点：

• Self-documenting: Clear docstrings explain purpose and usage自文档化：清晰的文档字符串解释了用途和用法
• Parameterized: Flexible inputs allow use in various scenarios参数化：灵活的输入允许在各种场景中使用
• Debuggable: Clean structure makes error identification easier可调试：清晰的结构使错误识别更容易
• Composable: Can be combined to create more complex skills可组合：可以组合起来创建更复杂的技能

SkillWeaver also creates more complex composite APIs by combining simpler ones, enabling hierarchical skill development.SkillWeaver 还通过结合更简单的 API 来创建更复杂的组合 API，从而实现分层技能开发。

Performance and Results性能与结果

SkillWeaver demonstrates significant performance improvements across multiple evaluation settings:SkillWeaver 在多个评估设置中展示了显著的性能提升：

1. WebArena Benchmark: On this standard web agent evaluation environment, equipping agents with SkillWeaver's synthesized APIs improved success rates from 25% to 38% (a relative gain of 52%). The improvement was even more dramatic for weaker base agents (40% to 133% relative improvement).WebArena 基准测试：在这个标准的 Web 智能体评估环境中，为智能体配备 SkillWeaver 合成的 API 后，成功率从 25% 提高到了 38%（相对提升 52%）。对于基础较弱的智能体，提升效果更为显著（相对提升 40% 到 133%）。

2. Real-World Websites: On tasks from the Online-Mind2Web dataset, agents equipped with SkillWeaver skills showed a 39.8% relative improvement in success rate.真实网站：在 Online-Mind2Web 数据集的任务上，配备 SkillWeaver 技能的智能体成功率相对提升了 39.8%。

3. Comparison with Human APIs: The performance of the automatically synthesized APIs approached the performance of human-crafted APIs extracted from official documentation, demonstrating the quality of SkillWeaver's synthesis process.与人工 API 的比较：自动合成 API 的性能接近从官方文档中提取的人工 API 的性能，证明了 SkillWeaver 合成过程的质量。

The results confirm that SkillWeaver's approach to autonomous skill discovery and synthesis provides substantial benefits for web agents across different websites and tasks.结果证实，SkillWeaver 的自主技能发现和合成方法为跨不同网站和任务的 Web 智能体提供了实质性的益处。

Real-World Applications现实世界应用

SkillWeaver demonstrates proficiency across diverse web scenarios, including:SkillWeaver 展示了在多种 Web 场景中的熟练度，包括：

1. E-commerce and Shopping: Skills for browsing products, adding items to carts, and completing checkout processes. The system can navigate shopping sites like the One-Stop Market shown in the following images:电子商务与购物：用于浏览产品、将商品添加到购物车以及完成结账流程的技能。该系统可以导航如后续图片中所示的 One-Stop Market 等购物网站：

2. Collaboration Tools: Skills for project management in environments like GitLab, including inviting team members and configuring repositories:协作工具：在 GitLab 等环境中进行项目管理的技能，包括邀请团队成员和配置存储库：

3. Information Retrieval: Skills for searching and extracting specific information from websites, like requesting quotes for car listings:信息检索：从网站搜索和提取特定信息的技能，例如为汽车列表请求报价：

4. Content Discovery: Skills for finding recipes and filtering by ingredients on cooking websites:内容发现：在烹饪网站上查找食谱并按配料进行筛选的技能：

These examples demonstrate SkillWeaver's versatility across different domains and task types, showing its potential for real-world applications in web automation.这些示例展示了 SkillWeaver 在不同领域和任务类型中的多功能性，体现了其在 Web 自动化现实应用中的潜力。

Key Innovations关键创新

SkillWeaver introduces several innovations that distinguish it from existing approaches:SkillWeaver 引入了多项使其区别于现有方法的创新：

1. Autonomous Skill Discovery: Unlike supervised approaches that require human-defined tasks, SkillWeaver autonomously explores and discovers potential skills based on website structure and functionality.自主技能发现：与需要人工定义任务的监督学习方法不同，SkillWeaver 基于网站结构和功能自主探索并发现潜在技能。

2. Structured Knowledge Representation: By representing skills as explicit APIs rather than implicit patterns, SkillWeaver creates a more interpretable and reusable knowledge base.结构化知识表示：通过将技能表示为明确的 API 而非隐式模式，SkillWeaver 创建了一个更具可解释性和可重用性的知识库。

3. Continuous Self-Improvement: The framework includes mechanisms for testing, debugging, and refining skills, enabling continuous improvement without human intervention.持续自我提升：该框架包含测试、调试和精炼技能的机制，无需人工干预即可实现持续改进。

4. Compositional Learning: SkillWeaver can combine existing skills to create more complex composite skills, enabling hierarchical learning.组合式学习：SkillWeaver 可以组合现有技能来创建更复杂的复合技能，从而实现分层学习。

5. Cross-Website Generalization: Skills learned on one website can often transfer to similar websites, improving general web navigation capabilities.跨网站泛化：在一个网站上学到的技能通常可以迁移到类似的网站，从而提高通用的 Web 导航能力。

Limitations and Future Work局限性与未来工作

Despite its promising results, SkillWeaver has several limitations:尽管结果令人振奋，但 SkillWeaver 仍存在一些局限性：

1. Parameter Generation Challenges: The system sometimes struggles to generate appropriate parameter values for testing, particularly for dynamic website data.参数生成挑战：系统有时难以生成合适的测试参数值，特别是对于动态网站数据而言。

2. API Robustness: Synthesized APIs may not handle all edge cases or website changes, requiring further refinement.API 稳健性：合成的 API 可能无法处理所有边缘情况或网站变更，需要进一步精炼。

3. Long-Horizon Tasks: While effective for short to medium-length tasks, the approach may need extensions to handle very complex, multi-step web interactions.长程任务：虽然对于中短程任务有效，但该方法可能需要扩展才能处理非常复杂的多步 Web 交互。

Future work directions include:未来的工作方向包括：

• Improving exploration strategies for more diverse skill discovery改进探索策略以实现更多样化的技能发现
• Developing better mechanisms for handling dynamic web content开发处理动态 Web 内容的更好机制
• Creating more sophisticated debugging and refinement processes创建更复杂的调试和精炼过程
• Extending the approach to multi-agent collaboration for complex tasks将方法扩展到用于复杂任务的多智能体协作

Conclusion结论

SkillWeaver represents a significant advance in autonomous web agents by enabling them to discover, encapsulate, and refine web skills without constant human supervision. By transforming successful interactions into structured APIs, the framework creates a growing library of reusable skills that improve agent performance across various websites and tasks.SkillWeaver 代表了自主 Web 智能体的重大进步，它使智能体能够在无需持续人工监督的情况下发现、封装和精炼 Web 技能。通过将成功交互转化为结构化 API，该框架创建了一个不断增长的可重用技能库，从而提升了智能体在各种网站和任务上的表现。

The approach demonstrates that web agents can indeed self-improve through autonomous exploration and practice, similar to how humans develop web navigation expertise. This capability has broad implications for web automation, digital assistants, and human-AI collaboration in digital environments.该方法证明，Web 智能体确实可以通过自主探索和实践实现自我提升，类似于人类发展 Web 导航专业知识的过程。这一能力对 Web 自动化、数字助理以及数字环境中的人机协作具有广泛意义。

As web interfaces continue to evolve, approaches like SkillWeaver that can autonomously adapt and learn new skills will be increasingly valuable for creating more capable and flexible AI systems that can navigate the digital world with greater independence and effectiveness.随着 Web 接口的不断演进，像 SkillWeaver 这样能够自主适应并学习新技能的方法，对于创建更强大、更灵活的 AI 系统将变得越来越有价值，使它们能够以更高的独立性和有效性在数字世界中导航。

Relevant Citations相关引用

Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024a. URLhttps://openreview.net/forum?id=oKn9c6ytLx.Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: A realistic web environment for building autonomous agents. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024a. URL https://openreview.net/forum?id=oKn9c6ytLx.

• This paper introduces WebArena, the primary benchmark environment used to evaluate SkillWeaver. It provides simulated websites and tasks for evaluating web agents and is a key component in assessing the effectiveness of SkillWeaver.这篇论文介绍了 WebArena，这是用于评估 SkillWeaver 的主要基准环境。它提供了用于评估 Web 智能体的模拟网站和任务，是评估 SkillWeaver 有效性的关键组成部分。

Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samual Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.),Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URLhttp://papers.nips.cc/paperfiles/paper/2023/hash/5950bf290a1570ea401bf98882128160-Abstract-DatasetsandBenchmarks.html.Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samual Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL http://papers.nips.cc/paperfiles/paper/2023/hash/5950bf290a1570ea401bf98882128160-Abstract-DatasetsandBenchmarks.html.

• Mind2Web is another important dataset used in the paper (although only the "Online" version is used). It provides realistic tasks for web agents on live websites, allowing for evaluating performance on complex and dynamic environments. The online version offers a more challenging evaluation for SkillWeaver than WebArena by using real websites.Mind2Web 是论文中使用的另一个重要数据集（尽管仅使用了“在线”版本）。它为真实网站上的 Web 智能体提供了现实任务，允许评估在复杂和动态环境中的性能。在线版本通过使用真实网站，为 SkillWeaver 提供了比 WebArena 更具挑战性的评估。

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi (Jim) Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models.Trans. Mach. Learn. Res., 2024, 2023a. URLhttps://api.semanticscholar.org/CorpusID:258887849.Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi (Jim) Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Trans. Mach. Learn. Res., 2024, 2023a. URL https://api.semanticscholar.org/CorpusID:258887849.

• Voyager is cited as related work demonstrating the effectiveness of self-improving agents in other complex environments (Minecraft). It inspires SkillWeaver's autonomous skill discovery and its approach to representing skills as APIs for enhanced decision-making.Voyager 被引用作为相关工作，展示了自我提升智能体在其他复杂环境（Minecraft）中的有效性。它启发了 SkillWeaver 的自主技能发现及其将技能表示为 API 以增强决策能力的方法。

Gabriel Sarch, Lawrence Jang, Michael J. Tarr, William W. Cohen, Kenneth Marino, and Katerina Fragkiadaki. Ical: Continual learning of multimodal agents by transforming trajectories into actionable insights.ArXiv, abs/2406.14596, 2024. URLhttps://api.semanticscholar.org/CorpusID:274466462.Gabriel Sarch, Lawrence Jang, Michael J. Tarr, William W. Cohen, Kenneth Marino, and Katerina Fragkiadaki. Ical: Continual learning of multimodal agents by transforming trajectories into actionable insights. ArXiv, abs/2406.14596, 2024. URL https://api.semanticscholar.org/CorpusID:274466462.

• ICAL is presented as an alternative method of continual learning for multimodal agents by using trajectories. SkillWeaver builds on ICAL by focusing on autonomous exploration and utilizing structured skills (APIs) rather than natural language routines. The comparison highlights the novelty of SkillWeaver.ICAL 被作为多模态智能体通过使用轨迹进行持续学习的替代方法提出。SkillWeaver 在 ICAL 的基础上，专注于自主探索并利用结构化技能（API）而非自然语言例程。这种比较突出了 SkillWeaver 的新颖性。