Tokenomics: Quantifying Where Tokens Are Used in Agentic Software EngineeringToken经济：量化智能体软件工程中Token的使用位置
Abstract.摘要。
LLM-based Multi-Agent (LLM-MA) systems are increasingly applied to automate complex software engineering tasks such as requirements engineering, code generation, and testing. However, their operational efficiency and resource consumption remain poorly understood, hindering practical adoption due to unpredictable costs and environmental impact. To address this, we conduct an analysis of token consumption patterns in an LLM-MA system within the Software Development Life Cycle (SDLC), aiming to understand where tokens are consumed across distinct software engineering activities. We analyze execution traces from 30 software development tasks performed by the ChatDev framework using a GPT-5 reasoning model mapping its internal phases to distinct development stages (Design, Coding, Code Completion, Code Review, Testing, and Documentation) to create a standardized evaluation framework. We then quantify and compare token distribution (input, output, reasoning) across these stages.基于LLM的多智能体（LLM-MA）系统越来越多地被应用于自动化复杂的软件工程任务，如需求工程、代码生成和测试。然而，其运行效率和资源消耗仍然鲜为人知，由于不可预测的成本和环境影响，阻碍了实际采用。为了解决这个问题，我们对软件开发生命周期（SDLC）中LLM-MA系统的Token消耗模式进行了分析，旨在了解不同的软件工程活动中的Token消耗位置。我们分析了使用GPT-5推理模型的ChatDev框架执行的30个软件开发任务的执行轨迹，将其内部阶段映射到不同的开发阶段（设计、编码、代码补全、代码审查、测试和文档），以创建一个标准化的评估框架。然后，我们量化并比较了这些阶段中Token的分布（输入、输出、推理）。
Our preliminary findings show that the iterative Code Review stage accounts for the majority of token consumption for an average of 59.4% of tokens. Furthermore, we observe that input tokens consistently constitute the largest share of consumption for an average of 53.9%, providing empirical evidence for potentially significant inefficiencies in agentic collaboration. Our results suggest that the primary cost of agentic software engineering lies not in initial code generation but in automated refinement and verification. Our novel methodology can help practitioners predict expenses and optimize workflows, and it directs future research toward developing more token-efficient agent collaboration protocols.我们的初步研究结果表明，迭代式的代码审查阶段占Token消耗的大部分，平均占Token的59.4%。此外，我们观察到输入Token始终占消耗的最大份额，平均为53.9%，为智能体协作中可能存在的显著低效提供了实证证据。我们的研究结果表明，智能体软件工程的主要成本不在初始代码生成，而在自动化的细化和验证。我们的新方法可以帮助从业者预测费用并优化工作流程，并引导未来研究朝着开发更Token高效的智能体协作协议的方向发展。
1. Introduction1. 引言
Large-scale software engineering increasingly explores LLM-Based Multi-Agent (LLM-MA) systems to automate complex tasks across the Software Development Life Cycle (SDLC) (He et al., 2025; Lu et al., 2025). These LLM-MA frameworks simulate human teams (e.g., product managers, architects, developers, testers) using specialized Large Language Model (LLM) agents that collaborate to design, code, and verify software. In principle, LLM-MA systems can improve autonomy and robustness by dividing work across agents (He et al., 2025). Prior work highlights that LLM-MA systems encourage divergent thinking (Liang et al., 2024), enhance reasoning and factuality (Du et al., 2024), and scale to problems beyond single-agent capacity (He et al., 2025). For Software Engineering (SE), this suggests LLM-MA systems could automate end-to-end workflows, from requirements to testing, in a unified manner (Hong et al., 2024; Qian et al., 2024).大规模软件工程越来越多地探索基于LLM的多智能体（LLM-MA）系统，以自动化复杂任务，贯穿整个软件开发生命周期（SDLC）（He等人，2025；Lu等人，2025）。这些LLM-MA框架通过专门化的大语言模型（LLM）智能体模拟人类团队（如产品经理、架构师、开发人员、测试人员），这些智能体协作设计、编码和验证软件。原则上，LLM-MA系统可以通过在智能体之间分配工作来提高自主性和鲁棒性（He等人，2025）。先前的工作强调，LLM-MA系统能激发发散性思维（Liang等人，2024），增强推理和事实性（Du等人，2024），并能扩展到超出单智能体能力的问题（He等人，2025）。对于软件工程（SE），这表明LLM-MA系统可以以统一的方式自动化从需求到测试的端到端工作流程（Hong等人，2024；Qian等人，2024）。
Recent studies have begun to analyze the behavior and efficiency of these systems. The AGENTTAXO framework (Wang et al., 2025) provided a taxonomy for dissecting token distribution in general LLM-MA systems, introducing the concept of a ”communication tax” to describe the overhead from inter-agent interactions. In addition, the MAST taxonomy of failures revealed that many issues in LLM-MA systems stem from systemic design and coordination challenges, such as step repetition or incomplete verification, rather than individual LLM limitations (Pan et al., 2025). While this prior work establishes essential taxonomies for understanding token distribution and systemic failure modes, it analyzes agent behavior in a general context. A significant knowledge gap exists regarding the resource efficiency of these systems when applied specifically to the unique, multi-stage SE workflows (Qiu et al., 2025). The ultimate question for practical adoption—”Where do the tokens go?”—remains unanswered in the SE domain.最近的研究开始分析这些系统的行为和效率。AGENTTAXO框架（Wang等人，2025）为剖析通用LLM-MA系统中的Token分布提供了一种分类法，引入了‘通信税’的概念来描述智能体间交互的开销。此外，MAST故障分类法揭示，LLM-MA系统中的许多问题源于系统设计和协调挑战，例如步骤重复或验证不完整，而非单个LLM的限制（Pan等人，2025）。虽然这些先前的工作为理解Token分布和系统性故障模式建立了基本分类法，但它们在通用情境下分析智能体行为。当这些系统专门应用于独特的、多阶段的SE工作流程时，关于其资源效率存在显著的知识空白（Qiu等人，2025）。实际采用的关键问题——‘Token都去哪了？’——在SE领域仍未得到解答。
Thus, in this paper, we introduce the term ”tokenomics” as the study of operational efficiency and resource consumption in LLM-MA systems. To our knowledge, this is the first study to conduct an empirical analysis of tokenomics in an SE context that examines execution traces of an SE-focused LLM-MA system through the lens of the SDLC. To guide our study, we focus on the following fundamental research question: What are the token consumption patterns of LLM-MA systems for software development tasks?因此，在本文中，我们引入‘tokenomics’一词，作为对LLM-MA系统中运营效率和资源消耗的研究。据我们所知，这是第一项在SE情境下对tokenomics进行实证分析的研究，该研究通过SDLC的视角检查了一个以SE为重点的LLM-MA系统的执行轨迹。为了指导我们的研究，我们聚焦于以下基本研究问题：LLM-MA系统在软件开发任务中的Token消耗模式是什么？
To answer this, we analyze the distribution of token consumption across distinct development stages, which we derive by mapping the internal phases of the multi-agent framework ChatDev (Qian et al., 2024). ChatDev simulates a virtual software company where multiple agent roles (e.g., programmer, tester) collaborate through multi-turn dialogues to complete the SDLC. Answering this question is the first step towards building economically and environmentally sustainable agentic SE systems.为了回答这个问题，我们分析了不同开发阶段的Token消耗分布，通过将多智能体框架ChatDev（Qian等人，2024）的内部阶段映射得到这些阶段。ChatDev模拟一个虚拟软件公司，其中多个智能体角色（如程序员、测试人员）通过多轮对话协作来完成SDLC。回答这个问题是构建经济且环境可持续的智能体SE系统的第一步。
This paper contributes an empirical analysis, a curated dataset of 30 execution traces, and a complete replication package111https://zenodo.org/records/17430187.本文贡献了一项实证分析、一个包含30次执行轨迹的精选数据集，以及一个完整的复现包111https://zenodo.org/records/17430187。
2. Study Design2. 研究设计
The goal of our study is to empirically investigate the distribution of token consumption within an LLM-MA system as it performs end-to-end software development tasks. To achieve this, we selected ChatDev (Qian et al., 2024) as our initial system for analysis. We make this choice because its ”chat chain” architecture represents a clear, sequential waterfall model (design coding testing), making its phases distinct and well-suited for mapping to software development stages. In addition, this framework is one of the most popular and highly cited open source frameworks (Wang et al., 2024; Guo et al., 2024; Qiu et al., 2025).我们研究的目标是实证调查LLM-MA系统在执行端到端软件开发任务时Token消耗的分布。为了实现这一目标，我们选择ChatDev（Qian等人，2024）作为我们初步分析的系统。我们之所以这样选择，是因为其‘聊天链’架构代表了一个清晰的顺序瀑布模型（设计→编码→测试），使其各个阶段清晰可辨，非常适合映射到软件开发阶段。此外，该框架是最受欢迎且被高度引用的开源框架之一（Wang等人，2024；Guo等人，2024；Qiu等人，2025）。
2.1. Dataset Curation2.1. 数据集整理
We executed ChatDev (Qian et al., 2024) on 30 distinct software development tasks, with the prompts sourced from the ProgramDev Dataset (Pan et al., 2025), which was used in the foundational MAST study (Pan et al., 2025). The selected prompts range from simple algorithms (e.g., Fibonacci number generation) to more complex applications (e.g., a chess game), ensuring task diversity. Recent work suggests that the number of reasoning tokens allocated by a model can serve as a proxy for task complexity (Shojaee et al., 2025). Our dataset exhibits a wide range in reasoning tokens consumed across the 30 tasks (from 17,280 to 40,000 tokens), which suggests a sufficient diversity in task complexity for this study.我们在30个不同的软件开发任务上执行了ChatDev（Qian等人，2024），提示词来自ProgramDev数据集（Pan等人，2025），该数据集曾用于基础的MAST研究（Pan等人，2025）。所选提示词从简单算法（如斐波那契数列生成）到更复杂的应用（如国际象棋游戏），确保了任务的多样性。近期研究表明，模型分配的推理Token数量可以作为任务复杂度的代理指标（Shojaee等人，2025）。我们的数据集在30个任务中消耗的推理Token范围很广（从17,280到40,000个Token），这表明本研究中的任务复杂度具有足够的多样性。
2.2. Model Selection2.2. 模型选择
The GPT-5 reasoning model was selected as the backbone for all agents. This decision was based on the popularity and recency of the model, its suitability for agentic use cases, and its strong reasoning capabilities, which are in line with the expectations of autonomous agents (He et al., 2025). As detailed in Table 1, the model version used is gpt-5-2025-08-07. The temperature parameter is not supported for this model, so the default value 1.0 was used.我们选择GPT-5推理模型作为所有智能体的骨干。这一决定基于该模型的热门度和时效性、其对智能体用例的适用性，以及其强大的推理能力，这符合自主智能体的预期（He等人，2025）。 如表1所示，使用的模型版本是gpt-5-2025-08-07。该模型不支持温度参数，因此使用了默认值1.0。
| Parameter | Value |
|---|---|
| Model Version | gpt-5-2025-08-07 |
| Temperature | 1.0 (default value; immutable) |
| Context Window | 400,000 tokens |
| Max Output Tokens | 128,000 tokens |
| Knowledge Cutoff | Sep 30, 2024 |
2.3. Analysis Pipeline2.3. 分析流程
To analyze the collected data, we designed and implemented a multi-step pipeline, illustrated in Figure 1.为了分析收集到的数据，我们设计并实现了一个多步骤流程，如图1所示。
Trace Collection. We instrumented ChatDev to log the complete execution trace for each of the 30 tasks, capturing every LLM call, including the prompt, response, and associated token counts (input, output, reasoning).轨迹收集。我们对ChatDev进行了插桩，以记录30个任务中每个任务的完整执行轨迹，捕获每次LLM调用，包括提示词、响应以及相关的Token计数（输入、输出、推理）。
Phase Mapping. A core methodological contribution of our work is the mapping of ChatDev’s internal, framework-specific phases to universally understood development stages. This abstraction allows for generalizable analysis and can be extended to other SE LLM-MA frameworks. The mapping used is detailed in Table 2.阶段映射。我们工作的一个核心方法学贡献是将ChatDev的内部、框架特定阶段映射到普遍理解的开发阶段。这种抽象允许进行可推广的分析，并且可以扩展到其他SE LLM-MA框架。所使用的映射详见表2。
Token Aggregation. Using this mapping, we wrote Python scripts to parse the traces collected and aggregate token counts for each of the development stages across all 30 runs, calculating totals and breaking them down by input, output, and reasoning tokens.Token聚合。利用这种映射，我们编写了Python脚本来解析收集到的轨迹，并聚合所有30次运行中每个开发阶段的Token计数，计算总数并按输入、输出和推理Token进行细分。
| Development Stage | ChatDev Phases | Description |
|---|---|---|
| Design | DemandAnalysis, LanguageChoose | These initial phases focus on understanding requirements and making high-level technical decisions. |
| Coding | Coding | This phase is directly involved in writing the initial source code. |
| Code Completion | CodeComplete | This phase completes any placeholder or incomplete code files left from the Coding phase. |
| Code Review | CodeReview | This phase involves an iterative dialogue between a programmer and code reviewer agent to review and modify/refine code. |
| Testing | Test | This phase explicitly focuses on dynamic system testing to locate and fix executability bugs. |
| Documentation | EnvironmentDoc, Reflection, Manual | These final phases generate user manuals and document required environment dependencies. |
3. Study Results3. 研究结果
In this section, we present the results of our research question. We present its motivation, the approach to answer the question, and the results.在本节中，我们展示研究问题的结果。我们介绍其动机、回答该问题的方法以及结果。
3.1. RQ: What are the token consumption patterns of LLM-MA systems for software development tasks?3.1. 研究问题：LLM-MA系统在软件开发任务中的Token消耗模式是什么？
Motivation. Understanding the token consumption patterns, or ”tokenomics,” of agentic SE systems is critical for their practical and sustainable adoption. High token usage translates directly to increased financial costs, energy consumption, and environmental impact. By identifying where tokens are consumed within the SDLC, we can create a ”cost map” that enables practitioners to predict expenses and optimize workflows. While prior work has analyzed general MAS behavior, there is a clear gap in understanding these efficiency patterns specifically within the context of software development (Qiu et al., 2025), which this RQ aims to address.动机。理解智能体SE系统的Token消耗模式（或称‘tokenomics’）对其实际和可持续采用至关重要。高Token使用直接转化为增加的财务成本、能源消耗和环境影响。通过识别SDLC中Token的消耗位置，我们可以创建一份‘成本地图’，使从业者能够预测费用并优化工作流程。虽然先前的工作分析了通用多智能体系统（MAS）的行为，但在软件开发情境下理解这些效率模式方面存在明显空白（Qiu等人，2025），本研究问题旨在填补这一空白。
Approach. To answer this question, we analyze the aggregated token data from the study pipeline described in Section 2. We focus on two primary dimensions:方法。为了回答这个问题，我们分析了第2节中描述的研究流程所得到的聚合Token数据。我们关注两个主要维度：
-
(1)
The distribution of total tokens across the mapped development stages (Design, Coding, etc.)总Token在映射的开发阶段（设计、编码等）中的分布
-
(2)
The ratio of input, output, and reasoning tokens within each stage.每个阶段中输入、输出和推理Token的比例。
Finding 1: The Code Review Stage Dominates Token Consumption. Our analysis reveals a highly uneven distribution of token usage across the development process. As shown in Figure 2, a clear hierarchy of token consumption emerges. In the figure, the ”n” value denotes the number of tasks (out of 30) where a specific phase was executed. This value is not always ”30”, as the agents within the multi-agent system autonomously decide which phases to execute, and not all phases are needed for every task. The error bars represent 1 standard deviation, indicating the variability in token consumption for each phase. The Code Review phase is the largest consumer, responsible for an average of 59.4% of tokens across all 30 tasks. The Code Completion phase, which occurred in 6 of the 30 tasks, was also costly, averaging of tokens in those runs. These two refinement-focused stages are followed by Documentation (avg. 20.1%) and Testing (avg. 10.3%), the latter of which occurred in 12 of the 30 tasks. In contrast, initial Coding (avg. 8.6%) and Design (avg. 2.4%) are remarkably inexpensive. This suggests that the primary cost of agentic software engineering lies not in initial code generation but in the iterative, conversational process of refinement and verification.发现1：代码审查阶段主导Token消耗。 我们的分析揭示了开发过程中Token使用的高度不均衡分布。如图2所示，Token消耗呈现清晰的层级。在图中，‘n’值表示执行了特定阶段的任务数量（共30个）。该值不总是‘30’，因为多智能体系统中的智能体自主决定执行哪些阶段，并非所有阶段都是每个任务所需的。误差条表示±1个标准差，指示每个阶段Token消耗的变异性。代码审查阶段是最大的消耗者，在所有30个任务中平均占Token的59.4%。代码补全阶段（在30个任务中出现6次）成本也很高，在这些运行中平均占Token的26.8%。这两个以细化为重点的阶段之后是文档（平均20.1%）和测试（平均10.3%），后者在30个任务中出现12次。相比之下，初始编码（平均8.6%）和设计（平均2.4%）非常便宜。这表明智能体软件工程的主要成本不在于初始代码生成，而在于迭代式、对话式的细化和验证过程。
Finding 2: Token Consumption is Dominated by Input Tokens. Across all phases except the Coding phase, we observe a consistent pattern where input tokens far exceed output and reasoning tokens. On average, the overall token usage for each task analyzed is composed of input tokens, output tokens, and reasoning tokens. This approximate 2:1 ratio of input to output tokens provides strong empirical evidence for the ”communication tax” identified in prior work (Wang et al., 2025), where agents repeatedly pass large contexts during their collaborative dialogue. This highlights a significant inefficiency in current agent collaboration protocols, where the majority of the tokens are spent on communicating context rather than generating novel output. This also suggests that the communication tax may be an inherent characteristic of conversational multi-agent architectures, a phenomenon which future work should investigate further.发现2：Token消耗以输入Token为主。除编码阶段外，所有阶段都有一个一致的模式，即输入Token远超过输出和推理Token。平均而言，每个任务的总Token使用由53.9%的输入Token、24.4%的输出Token和21.6%的推理Token组成。这一大约2:1的输入输出Token比例，为先前工作中识别的‘通信税’（Wang等人，2025）提供了强有力的实证证据，即智能体在协作对话中反复传递大型上下文。这凸显了当前智能体协作协议中的显著低效，即大部分Token用于传达上下文而非生成新的输出。这也表明，通信税可能是对话式多智能体架构的固有特征，未来工作应进一步研究这一现象。
Finding 3: Software Development Stages Exhibit Distinct Tokenomic Profiles. A deeper look at the token ratios per phase, detailed in Table 3, reveals unique patterns for different software engineering activities. The Coding phase is a notable outlier, being output-heavy (58% output vs. 6.9% input). This is intuitive, as it involves generating verbose source code from a more concise design specification. In contrast, verification phases like Code Review, and documentation phases are input-heavy (51.4% and 80.2% input, respectively). These phases consume large amounts of existing code as context to produce small, analytical outputs. These distinct profiles provide a ”cost map” for different engineering activities, enabling practitioners to better predict expenses and identify opportunities for process optimization.发现3：软件开发阶段表现出不同的Token经济特征。对每个阶段Token比例的深入观察（详见表3）揭示了不同软件工程活动的独特模式。编码阶段是一个显著的异常值，以输出为主（58%的输出对比6.9%的输入）。这很直观，因为它涉及从更简洁的设计说明生成冗长的源代码。相比之下，像代码审查这样的验证阶段和文档阶段则以输入为主（输入分别占51.4%和80.2%）。这些阶段消耗大量现有代码作为上下文，以产生小规模的分析性输出。这些不同的特征为不同的工程活动提供了‘成本地图’，使从业者能够更好地预测费用并识别流程优化的机会。
| Development Stage | Avg Input % | Avg Output % | Avg Reasoning % |
|---|---|---|---|
| Design | 60.4 | 3.6 | 36.0 |
| Coding | 6.9 | 58.0 | 35.1 |
| Code Completion | 47.7 | 41.7 | 10.5 |
| Code Review | 51.4 | 24.7 | 23.9 |
| Testing | 60.8 | 20.7 | 18.4 |
| Documentation | 80.2 | 8.3 | 11.5 |
| Overall (per task) | 53.9 | 24.4 | 21.6 |
Answer to RQ: With the ChatDev LLM-MA system, token consumption is heavily concentrated in the Code Review stage of the SDLC. Moreover, token consumption is dominated by input tokens, reflecting a potentially significant communication tax, with different development stages exhibiting unique tokenomic profiles corresponding to the nature of the software engineering task (e.g., planning, reasoning, or verification). 研究问题解答：在ChatDev LLM-MA系统中，Token消耗高度集中在SDLC的代码审查阶段。此外，Token消耗以输入Token为主，反映了可能显著的通信税，不同的开发阶段表现出与软件工程任务性质（如规划、推理或验证）相对应的独特Token经济特征。
4. Discussion4. 讨论
Our preliminary results offer an initial ”cost map” of agentic software development, with several implications for practitioners and researchers.我们的初步结果为智能体软件开发提供了一份初始的‘成本地图’，对从业者和研究人员有几个启示。
The immense token cost of the Code Review phase can be interpreted as the ”Cost of Conversation.” This is a direct consequence of the inherent conversational architecture of LLM-MA systems, where agents iteratively pass the full code context back and forth to refine it. This suggests that current agentic collaboration protocols for verification are highly inefficient, consuming vast resources to perform tasks that might involve minor corrections. This aligns with findings from the MAST taxonomy (Pan et al., 2025), where failures related to verification and step repetition are common, suggesting that high token usage may be a symptom of the agentic system’s attempt to overcome these inherent coordination challenges through brute-force dialogue.代码审查阶段的巨大Token成本可以被解释为‘对话成本’。这是LLM-MA系统固有的对话架构的直接后果，智能体反复传递完整的代码上下文来进行细化。这表明当前用于验证的智能体协作协议效率极低，消耗大量资源来执行可能只涉及微小修正的任务。这与MAST分类法（Pan等人，2025）的发现一致，其中与验证和步骤重复相关的故障很常见，这表明高Token使用可能是智能体系统试图通过蛮力对话来克服这些固有协调挑战的症状。
For practitioners, our findings provide a basis for cost prediction and process optimization. The distinct tokenomic profiles imply that the cost of an agent-driven project can be estimated based on the type of work required. For example, greenfield projects with heavy initial coding will have a different cost structure than projects focused on refactoring and debugging existing code, which will be dominated by the expensive, input-heavy code review cycle. This insight can inform design decisions, such as integrating a ”human-in-the-loop” checkpoint before the Code Review phase to prevent costly iterative loops (Navneet and Chandra, 2025), thereby maximizing both economic and computational efficiency.对于从业者，我们的发现为成本预测和流程优化提供了基础。不同的Token经济特征意味着，智能体驱动项目的成本可以根据所需工作类型来估算。例如，需要大量初始编码的新项目，其成本结构将不同于专注于重构和调试现有代码的项目，后者将由昂贵的、以输入为主的代码审查周期主导。这一见解可以为设计决策提供信息，例如在代码审查阶段之前集成‘人在环路’检查点，以防止昂贵的迭代循环（Navneet和Chandra，2025），从而最大化经济和计算效率。
For the research community, our results present a clear challenge and a potential solution. The challenge is to design more token-efficient collaboration protocols for verification and refinement, moving beyond naive full-context passing. In addition, there is a clear need for a standardized, comprehensive evaluation framework (Li et al., 2025). This framework can serve as a common ground to benchmark and compare the efficiency of different LLM-MA architectures (e.g., ChatDev’s hierarchical, conversational workflow vs. MetaGPT’s SOP-based assembly line) in future work, providing a ”Rosetta Stone” to translate framework-specific operations into universal software engineering activities.对于研究社区，我们的结果提出了一个明确的挑战和一个潜在的解决方案。挑战是为验证和细化设计更Token高效的协作协议，超越天真的全上下文传递。此外，明确需要一个标准化、全面的评估框架（Li等人，2025）。该框架可以作为共同基础，在未来工作中基准测试和比较不同LLM-MA架构（例如，ChatDev的分层对话工作流与MetaGPT基于SOP的装配线）的效率，提供一块‘罗塞塔石碑’，将框架特定操作转化为通用的软件工程活动。
5. Threats to Validity5. 有效性威胁
There are a few important limitations to our work that need to be considered when interpreting our findings. Firstly, our analysis is based on a single LLM-MA system (ChatDev) and a single LLM (GPT-5 Reasoning Model). The observed token consumption patterns may differ in other LLM-MA architectures or with other LLMs that have different token efficiencies (Wang et al., 2025). Secondly, the 30 software development tasks, while diverse, may not represent all possible software development scenarios and complexities. The size of the curated dataset is a direct consequence of the current lack of public, large-scale benchmarks for SE-specific agent traces (Yan et al., 2025), which makes data curation a time-consuming and costly process. Thirdly, some development stages were executed in only a small subset of the 30 tasks. For instance, the Code Completion () and Testing () phases were triggered infrequently by the agentic system. The conclusions drawn about the tokenomic profiles of these specific stages are based on a small sample, which may not be representative and may limit the generalizability of those particular findings. Finally, our proposed mapping of ChatDev’s internal phases to software development stages is an abstraction. While we believe it is a logical and useful one for creating a standardized evaluation framework, it represents one of several possible mappings of the agent’s activities.在解释我们的发现时，需要考虑我们工作的几个重要局限。首先，我们的分析基于单一的LLM-MA系统（ChatDev）和单一的LLM（GPT-5推理模型）。观察到的Token消耗模式可能在其他LLM-MA架构或具有不同Token效率的其他LLM中有所不同（Wang等人，2025）。其次，30个软件开发任务虽然多样，但可能无法代表所有可能的软件开发场景和复杂性。精选数据集的大小直接源于目前缺乏针对SE特定智能体轨迹的公开大规模基准（Yan等人，2025），这使得数据整理成为一个耗时且昂贵的过程。第三，某些开发阶段仅在30个任务中的一小部分被执行。例如，代码补全（n=6）和测试（n=12）阶段很少被智能体系统触发。关于这些特定阶段的Token经济特征得出的结论基于小样本，可能不具有代表性，并可能限制这些特定发现的普遍性。最后，我们提出的将ChatDev内部阶段映射到软件开发阶段是一种抽象。虽然我们认为这是一种逻辑上且有用于创建标准化评估框架的映射，但它代表了智能体活动多种可能映射中的一种。
6. Conclusion and Future Work6. 结论与未来工作
This work-in-progress paper sets out to answer ”where do the tokens go?” in agentic software engineering. Our preliminary empirical study using the ChatDev framework reveals that the answer is not straightforward. The costs are not evenly distributed but are overwhelmingly concentrated in the iterative, conversational phase of code review. We also found that input tokens, comprising the ”communication tax,” form the bulk of the token usage, highlighting a key area for future optimization.这篇进行中的论文旨在回答智能体软件工程中‘Token都去哪了？’的问题。我们使用ChatDev框架进行的初步实证研究表明，答案并不简单。成本并非均匀分布，而是绝大多数集中在迭代式的对话式代码审查阶段。我们还发现，构成‘通信税’的输入Token是Token使用的主体，突出了未来优化的一个关键领域。
This study lays the groundwork for a comprehensive research agenda. Future work should focus on:这项研究为全面的研究议程奠定了基础。未来的工作应侧重于：
-
(1)
Expanding our dataset with more tasks to ensure better generalizability.扩展我们的数据集，加入更多任务，以确保更好的普遍性。
-
(2)
Extending the analysis to other LLMs to understand model-specific effects.将分析扩展到其他LLM，以理解特定模型的影响。
-
(3)
Extending the analysis to other LLM-MA systems to perform a comparative study of how architectural differences impact tokenomics.将分析扩展到其他LLM-MA系统，以进行架构差异如何影响Token经济的比较研究。
-
(4)
Investigating the relationship between token consumption patterns and failure modes.研究Token消耗模式与故障模式之间的关系。
-
(5)
Further developing and validating our development stage mapping as a robust, universal framework for benchmarking SE agent efficiency.进一步开发和验证我们的开发阶段映射，作为基准测试SE智能体效率的稳健、通用框架。
References
- Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, pp. 11733–11763. Cited by: §1.
- Large language model based multi-agents: a survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pp. 8048–8057. Cited by: §2.
- LLM-based multi-agent systems for software engineering: literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology 34 (5), pp. 1–30. Cited by: §1, §2.2.
- MetaGPT: meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1.
- Beyond single-turn: a survey on multi-turn interactions with large language models. arXiv preprint arXiv:2504.04717. Cited by: §4.
- Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 17889–17904. Cited by: §1.
- Exploring autonomous agents: a closer look at why they fail when completing tasks. arXiv preprint arXiv:2508.13143. Cited by: §1.
- Rethinking autonomy: preventing failures in ai-driven software engineering. arXiv preprint arXiv:2508.11824. Cited by: §4.
- Why do multiagent systems fail?. In ICLR 2025 Workshop on Building Trust in Language Models and Applications, External Links: Link Cited by: §1, §2.1, §4.
- ChatDev: communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15174–15186. Cited by: §1, §1, §2.1, §2.
- Co-saving: resource aware multi-agent collaboration for software development. arXiv preprint arXiv:2505.21898. Cited by: §1, §2, §3.1.
- The illusion of thinking: understanding the strengths and limitations of reasoning models via the lens of problem complexity. arXiv preprint arXiv:2506.06941. Cited by: §2.1.
- A survey on large language model based autonomous agents. Frontiers of Computer Science 18 (6), pp. 186345. Cited by: §2.
- AgentTaxo: dissecting and benchmarking token distribution of LLM multi-agent systems. In ICLR 2025 Workshop on Foundation Models in the Wild, External Links: Link Cited by: §1, §3.1, §5.
- Beyond self-talk: a communication-centric survey of llm-based multi-agent systems. arXiv preprint arXiv:2502.14321. Cited by: §5.