How OpenTelemetry traces LLM Calls, Agent Reasoning, and MCP ToolsOpenTelemetry 如何追踪 LLM 调用、Agent 推理和 MCP 工具
GreptimeGreptime
OpenTelemetry GenAI Semantic Conventions standardize observability for LLM apps, agent orchestration, MCP tool calling, content capture, and quality evaluation. This article goes through all six layers: what each one defines, why it’s designed that way, and how mature it is.OpenTelemetry GenAI 语义约定标准化了 LLM 应用、Agent 编排、MCP 工具调用、内容捕获和质量评估的可观测性。本文涵盖所有六个层次：每个层次定义了什么、为何如此设计以及成熟度如何。
Observability for LLM applications doesn’t work like traditional microservices. An LLM call produces far more telemetry than an HTTP request, but the harder problem is the shape of that data. Prompts and completions are large text blobs. Tool call parameters take a different structure every time. An agent’s multi-step reasoning can’t be captured in a fixed schema. And beyond the usual “which model was called and how long did it take,” you also need to know how many tokens were consumed, how much the call cost, and whether the answer was any good.LLM 应用的可观测性与传统微服务不同。一次 LLM 调用产生的遥测数据远多于一次 HTTP 请求，但更棘手的是数据的形态。提示和补全是大型文本块。工具调用参数每次结构都不同。Agent 的多步推理无法用固定模式捕获。除了常规的“调用了哪个模型、耗时多久”，你还需要知道消耗了多少 token、调用成本多少以及答案质量如何。
Traditional OTel semantic conventions don’t cover any of this. http.request.method and db.system.name are meaningless for LLM calls. The community needed a dedicated spec.传统的 OTel 语义约定无法涵盖这些。http.request.method 和 db.system.name 对 LLM 调用毫无意义。社区需要一个专用规范。
In April 2024, OpenTelemetry formed the GenAI Special Interest Group (GenAI SIG) under the Semantic Conventions SIG[1]. The original scope was LLM client call tracing. Since then it has expanded to cover agent orchestration, MCP tool calling, content capture, and quality evaluation: six layers in all. As of this writing, the OTel docs site shows Semantic Conventions v1.41.0[2], with the latest GitHub release at v1.41.1 (a k8s codegen fix only, no GenAI changes)[3].2024 年 4 月，OpenTelemetry 在语义约定 SIG[1] 下成立了 GenAI 特别兴趣小组 (GenAI SIG)。最初的范围是 LLM 客户端调用追踪。此后扩展到涵盖 Agent 编排、MCP 工具调用、内容捕获和质量评估：共六个层次。截至本文撰写时，OTel 文档站点显示语义约定 v1.41.0[2]，最新 GitHub 发布为 v1.41.1（仅 k8s codegen 修复，无 GenAI 变更）[3]。
Each layer below covers what the spec defines, why it’s designed that way, and where it stands in maturity. If you read our earlier piece on Agent Observability, this is the spec companion: how the community is standardizing around the challenges that article described.以下每个层次涵盖规范定义的内容、设计原因以及成熟度。如果你读过我们之前关于 Agent 可观测性的文章，这是规范配套：社区如何围绕那篇文章描述的挑战进行标准化。
How the Spec Evolved: From Early Days to v1.41规范如何演变：从早期到 v1.41
When GenAI SIG kicked off in April 2024, the goal was straightforward: define a standard set of attributes for LLM client calls (model name, token usage, latency) so that different instrumentation libraries produce consistent telemetry. Early versions introduced gen_ai.system, gen_ai.request.model, and a pair of histogram metrics.当 GenAI SIG 于 2024 年 4 月启动时，目标很直接：为 LLM 客户端调用（模型名称、token 用量、延迟）定义一组标准属性，以便不同插桩库产生一致的遥测数据。早期版本引入了 gen_ai.system、gen_ai.request.model 和一对直方图指标。
v1.37 was the turning point. It overhauled how chat history is recorded[4]. The old per-message events (one event per message) were replaced by three aggregated attributes (gen_ai.system_instructions, gen_ai.input.messages, gen_ai.output.messages) stored either on the span or in a new gen_ai.client.inference.operation.details event. The reason was practical: per-message events flooded multi-turn conversations with fine-grained events that were painful to query and correlate.v1.37 是转折点。它彻底改革了聊天历史的记录方式[4]。旧的每条消息事件（每条消息一个事件）被三个聚合属性（gen_ai.system_instructions、gen_ai.input.messages、gen_ai.output.messages）取代，这些属性存储在 span 上或新的 gen_ai.client.inference.operation.details 事件中。原因是实际的：每条消息事件在多轮对话中产生大量细粒度事件，难以查询和关联。
Every subsequent release has included GenAI changes:后续每个版本都包含 GenAI 变更：
As of May 2026, the GenAI and MCP semantic conventions remain in Development status[2:1]. The docs say it plainly: “This transition plan will be updated to include stable version before the GenAI conventions are marked as stable.” No public timeline for stabilization. The 2026 Semantic Conventions Roadmap[9] is collecting proposals from each sub-SIG, but nothing’s committed.截至 2026 年 5 月，GenAI 和 MCP 语义约定仍处于开发状态[2:1]。文档明确说明：“此过渡计划将在 GenAI 约定标记为稳定之前更新为稳定版本。”没有公开的稳定化时间表。2026 年语义约定路线图[9]正在收集每个子 SIG 的提案，但尚未承诺任何内容。
Attribute names and structures may still change. The core concepts have settled though, and the spec provides OTEL_SEMCONV_STABILITY_OPT_IN to manage version transitions. Building on the spec today is a reasonable bet.属性名称和结构可能仍会变化。但核心概念已经稳定，规范提供了 OTEL_SEMCONV_STABILITY_OPT_IN 来管理版本过渡。基于当前规范构建是合理的。
Layer 1: Client Spans — Standardizing Model Calls第一层：客户端 Span — 标准化模型调用
This is the base layer of the spec. It defines the spans produced when application code calls a GenAI model[10].这是规范的基础层。它定义了应用代码调用 GenAI 模型时产生的 span[10]。
Inference推理
Each LLM call generates a span with gen_ai.operation.name set to chat, text_completion, or generate_content (for multimodal). Span kind is CLIENT, since models typically run on remote services.每次 LLM 调用生成一个 span，其 gen_ai.operation.name 设置为 chat、text_completion 或 generate_content（用于多模态）。Span 类型为 CLIENT，因为模型通常运行在远程服务上。
Core attributes:核心属性：
Here’s what this looks like in a trace viewer (Jaeger, Grafana Tempo, etc.):在追踪查看器（Jaeger、Grafana Tempo 等）中看起来如下：
{
"operationName": "chat gpt-4o-mini",
"spanKind": "CLIENT",
"duration": "1.23s",
"attributes": {
"gen_ai.operation.name": "chat",
"gen_ai.provider.name": "openai",
"gen_ai.request.model": "gpt-4o-mini",
"gen_ai.response.model": "gpt-4o-mini-2024-07-18",
"gen_ai.usage.input_tokens": 142,
"gen_ai.usage.output_tokens": 87,
"gen_ai.response.finish_reasons": ["stop"],
"server.address": "api.openai.com",
"server.port": 443
}
}Note: This is a logical view for readability. The actual OTLP wire format uses
KeyValuearrays with typed values (stringValue,intValue, etc.) and nanosecond Unix timestamps.注意：这是为可读性而设计的逻辑视图。实际的 OTLP 线缆格式使用带有类型化值（stringValue、intValue 等）和纳秒 Unix 时间戳的 KeyValue 数组。
server.address and server.port are general networking attributes. They're not part of the GenAI spec, but instrumentation typically emits them anyway.server.address 和 server.port 是通用网络属性。它们不属于 GenAI 规范，但插桩通常会发出它们。
provider.name and request.model are intentionally separate. The same model name can be accessed through different providers (Azure OpenAI and OpenAI direct both serve GPT-4o), and provider.name determines which provider-specific attributes apply.provider.name 和 request.model 是故意分开的。相同的模型名称可以通过不同提供商访问（Azure OpenAI 和 OpenAI 直连都提供 GPT-4o），而 provider.name 决定了哪些提供商特定属性适用。
request.model vs response.model matters in practice too: you request gpt-4o, but the response might come from gpt-4o-2024-08-06. For fine-tuned models, response.model should be more specific than the base name.request.model 与 response.model 在实践中也很重要：你请求 gpt-4o，但响应可能来自 gpt-4o-2024-08-06。对于微调模型，response.model 应比基础名称更具体。
Embeddings and Retrievals嵌入和检索
Embeddings (gen_ai.operation.name=embeddings) cover vector embedding operations, with gen_ai.embeddings.dimension.count recording the vector dimension. Retrievals cover the retrieval step in RAG pipelines.嵌入（gen_ai.operation.name=embeddings）涵盖向量嵌入操作，gen_ai.embeddings.dimension.count 记录向量维度。检索涵盖 RAG 流水线中的检索步骤。
One-Line Integration一行集成
With Python + OpenAI SDK, instrumentation takes one line[11]:使用 Python + OpenAI SDK，插桩只需一行[11]：
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
OpenAIInstrumentor().instrument()
# OpenAI SDK calls will emit semconv-compliant spans and metrics.
# Prompt/completion events require content capture to be enabled explicitly.
client.chat.completions.create(model="gpt-4o-mini", messages=[...])opentelemetry-instrumentation-openai-v2 is the most mature GenAI instrumentation. Anthropic, AWS Bedrock, and others are covered by community libraries.opentelemetry-instrumentation-openai-v2 是最成熟的 GenAI 插桩。Anthropic、AWS Bedrock 等由社区库覆盖。
Layer 2: Agent & Workflow Spans — Beyond Microservices第二层：Agent 和工作流 Span — 超越微服务
This is where GenAI semantic conventions diverge most from traditional OTel. Distributed tracing has HTTP spans, RPC spans, and DB spans, but nothing for “agent invocation.” The GenAI spec introduces a new set of operation types[12].这是 GenAI 语义约定与传统 OTel 差异最大的地方。分布式追踪有 HTTP span、RPC span 和 DB span，但没有“Agent 调用”的对应项。GenAI 规范引入了一组新的操作类型[12]。
create_agentcreate_agent
Describes agent creation, typically for remote agent services (OpenAI Assistants API, AWS Bedrock Agents). Span kind is CLIENT.描述 Agent 创建，通常用于远程 Agent 服务（OpenAI Assistants API、AWS Bedrock Agents）。Span 类型为 CLIENT。
Span: create_agent support-router
Kind: CLIENT
Attributes:
gen_ai.operation.name = create_agent
gen_ai.agent.name = support-router
gen_ai.provider.name = openaiinvoke_agentinvoke_agent
Invokes an agent to perform a task. v1.41 explicitly splits two scenarios[8:1]: CLIENT for remote calls, INTERNAL for local framework execution (e.g. LangGraph running agent logic in-process).调用 Agent 执行任务。v1.41 明确拆分为两种场景[8:1]：CLIENT 用于远程调用，INTERNAL 用于本地框架执行（例如 LangGraph 在进程中运行 Agent 逻辑）。
invoke_workflowinvoke_workflow
Describes execution of a predefined workflow (added in v1.41[8:2]). Agents reason autonomously; workflows follow predetermined paths.描述预定义工作流的执行（在 v1.41 中新增[8:2]）。Agent 自主推理；工作流遵循预定路径。
execute_toolexecute_tool
Tool execution span, kind INTERNAL. Starting with v1.41, the tool name must appear in the span name (execute_tool {gen_ai.tool.name})[8:3]. gen_ai.tool.call.arguments and gen_ai.tool.call.result are recorded only when privacy policies permit.工具执行 span，类型为 INTERNAL。从 v1.41 开始，工具名称必须出现在 span 名称中（execute_tool {gen_ai.tool.name}）[8:3]。gen_ai.tool.call.arguments 和 gen_ai.tool.call.result 仅在隐私策略允许时记录。
Why These Span Types Matter为什么这些 Span 类型很重要
Our Agent Observability article discussed a core challenge: the agent’s reasoning process shows up as a black box in traces. Agent span conventions crack that open:我们的 Agent 可观测性文章讨论了一个核心挑战：Agent 的推理过程在追踪中显示为黑盒。Agent span 约定打开了这个黑盒：
invoke_agent research-assistant (INTERNAL)
├── chat gpt-4o (CLIENT) ← Model decides to search
├── execute_tool web_search (INTERNAL) ← Search executed
├── chat gpt-4o (CLIENT) ← Continues reasoning with results
├── execute_tool summarize (INTERNAL) ← Summarization
└── chat gpt-4o (CLIENT) ← Generates final answerEvery step carries standardized attributes, and any compatible backend (Jaeger, Tempo, Datadog) can render the structure correctly.每一步都携带标准化属性，任何兼容的后端（Jaeger、Tempo、Datadog）都能正确渲染结构。
Layer 3: MCP Semantic Conventions — Fixing Broken Traces第三层：MCP 语义约定 — 修复断裂的追踪
Model Context Protocol (MCP) spread fast in 2025, but brought an observability problem: traces from the agent side and the MCP server side are disconnected.模型上下文协议 (MCP) 在 2025 年迅速普及，但带来了可观测性问题：Agent 端和 MCP 服务器端的追踪是断开的。
Glama’s analysis laid it out clearly[13]: the agent produces Trace A, the MCP server produces Trace B, no context propagation between them. The OTel MCP semantic conventions, introduced in v1.39[6:1], fix this[14].Glama 的分析清楚地指出了这一点[13]：Agent 产生追踪 A，MCP 服务器产生追踪 B，两者之间没有上下文传播。在 v1.39[6:1] 中引入的 OTel MCP 语义约定修复了这个问题[14]。
Core Design核心设计
MCP runs on JSON-RPC, but the spec recommends MCP conventions over generic RPC semantic conventions. MCP spans carry context that generic RPC conventions miss, such as session and tool call details.MCP 运行在 JSON-RPC 上，但规范建议使用 MCP 约定而非通用 RPC 语义约定。MCP span 携带通用 RPC 约定遗漏的上下文，例如会话和工具调用细节。
Client span example (stdio transport):客户端 span 示例（stdio 传输）：
Span: tools/call get-weather
Kind: CLIENT
Attributes:
gen_ai.operation.name = execute_tool
mcp.method.name = tools/call
mcp.session.id = session-xyz
mcp.protocol.version = 2025-03-26
gen_ai.tool.name = get-weather
jsonrpc.request.id = 42
network.transport = pipe # stdio maps to pipeFor HTTP transport, use network.transport = tcp (or quic) with network.protocol.name = http[14:1].对于 HTTP 传输，使用 network.transport = tcp（或 quic）和 network.protocol.name = http[14:1]。
When both sides propagate W3C Trace Context, the server span nests under the client span, and trace continuity is preserved across the protocol boundary.当双方传播 W3C 追踪上下文时，服务器 span 嵌套在客户端 span 下，追踪连续性在协议边界得以保持。
Compatibility with execute_tool与 execute_tool 的兼容性
The spec handles deduplication: if MCP instrumentation detects that an outer GenAI instrumentation already tracks tool execution, it enriches the existing span with MCP-specific attributes (mcp.method.name, mcp.session.id, etc.) instead of creating a duplicate.规范处理了去重：如果 MCP 插桩检测到外部 GenAI 插桩已追踪工具执行，它会用 MCP 特定属性（mcp.method.name、mcp.session.id 等）丰富现有 span，而不是创建重复 span。
A complete agent + MCP call chain[14:2]:完整的 Agent + MCP 调用链[14:2]：
invoke_agent weather-forecast-agent (INTERNAL)
├── chat {model} (CLIENT) ← GenAI model
├── tools/call get-weather (CLIENT) ← MCP client
│ └── tools/call get-weather (SERVER) ← MCP server
└── chat {model} (CLIENT) ← GenAI modelMCP-Specific MetricsMCP 特定指标
Four MCP metrics: mcp.client.operation.duration / mcp.server.operation.duration (operation latency) and mcp.client.session.duration / mcp.server.session.duration (session lifetime).四个 MCP 指标：mcp.client.operation.duration / mcp.server.operation.duration（操作延迟）和 mcp.client.session.duration / mcp.server.session.duration（会话生命周期）。
Layer 4: Events and Content Capture — Balancing Privacy and Observability第四层：事件和内容捕获 — 平衡隐私与可观测性
Traditional OTel HTTP spans rarely need to worry about “should we record the request body?” LLM applications are different: prompt and completion content is both the most valuable debugging data and the most sensitive.传统 OTel HTTP span 很少需要担心“是否应该记录请求体”。LLM 应用则不同：提示和补全内容既是最有价值的调试数据，也是最敏感的数据。
Two Core Events两个核心事件
gen_ai.client.inference.operation.details (added in v1.37)[15]: records full input and output for a GenAI call. It's opt-in, and backends can process it as events/logs, decoupled from trace lifecycle and storage policies.gen_ai.client.inference.operation.details（在 v1.37 中新增）[15]：记录 GenAI 调用的完整输入和输出。它是可选的，后端可以将其作为事件/日志处理，与追踪生命周期和存储策略解耦。
gen_ai.evaluation.result: records quality evaluation results via gen_ai.evaluation.score.value and gen_ai.evaluation.score.label[15:1]. A relevancy evaluator might return score.value=0.85, score.label="relevant".gen_ai.evaluation.result：通过 gen_ai.evaluation.score.value 和 gen_ai.evaluation.score.label[15:1] 记录质量评估结果。相关性评估器可能返回 score.value=0.85，score.label="relevant"。
Here’s the operation.details event with content capture enabled:以下是启用内容捕获的 operation.details 事件：
{
"eventName": "gen_ai.client.inference.operation.details",
"attributes": {
"gen_ai.system_instructions": [
{"type": "text", "content": "You are a helpful customer support agent."}
],
"gen_ai.input.messages": [
{
"role": "user",
"parts": [{"type": "text", "content": "Where is my order #12345?"}]
}
],
"gen_ai.output.messages": [
{
"role": "assistant",
"parts": [{"type": "text", "content": "Your order #12345 shipped this morning and should arrive tomorrow."}],
"finish_reason": "stop"
}
]
}
}When recorded on the operation.details event, messages follow the spec's JSON schema in structured form[15:2]. On span attributes, where backends lack structured attribute support, serialized JSON strings work too.当记录在 operation.details 事件上时，消息遵循规范的 JSON 模式以结构化形式呈现[15:2]。在 span 属性上，当后端缺乏结构化属性支持时，也可以使用序列化的 JSON 字符串。
Content is absent by default. Many instrumentations gate it behind OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true. In production, most teams end up on mode three: external storage with a reference URL on the span.默认情况下内容不存在。许多插桩通过 OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true 来控制。在生产环境中，大多数团队最终采用第三种模式：外部存储，并在 span 上提供引用 URL。
Three Content Recording Modes三种内容记录模式
The spec defines three approaches[10:1]:规范定义了三种方法[10:1]：
Not recorded (default). Content capture is off.不记录（默认）。内容捕获关闭。
On span attributes. gen_ai.input.messages and gen_ai.output.messages as span attributes. Convenient, but size-limited and visible to anyone with trace access.在 span 属性上。gen_ai.input.messages 和 gen_ai.output.messages 作为 span 属性。方便，但有大小限制，且任何有追踪访问权限的人都能看到。
External storage + span reference. Full content in external storage (S3, GreptimeDB, etc.), span holds only a reference URL. Independent IAM and retention policies. The spec recommends this for production with significant telemetry volume or sensitive data[10:2].外部存储 + span 引用。完整内容存储在外部存储（S3、GreptimeDB 等）中，span 仅持有引用 URL。独立的 IAM 和保留策略。规范建议在遥测数据量大或包含敏感数据时用于生产环境[10:2]。
Layer 5: Metrics — The Two Essential Client Histograms第五层：指标 — 两个基本的客户端直方图
The GenAI spec defines multiple metrics on both client and server sides. The two most used client histograms[16]:GenAI 规范在客户端和服务器端定义了多个指标。最常用的两个客户端直方图[16]：
gen_ai.client.operation.durationgen_ai.client.operation.duration
End-to-end latency per GenAI operation, in seconds. Dimensions: gen_ai.operation.name, gen_ai.request.model, gen_ai.provider.name.每次 GenAI 操作的端到端延迟，单位为秒。维度：gen_ai.operation.name、gen_ai.request.model、gen_ai.provider.name。
gen_ai.client.token.usagegen_ai.client.token.usage
Token consumption per operation, unit {token}. Recommended bucket boundaries follow exponential growth: [1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864], covering 1 token to 67M tokens.每次操作的 token 消耗，单位为 {token}。推荐的桶边界遵循指数增长：[1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864]，覆盖 1 到 6700 万 token。
Two counting rules worth knowing. When a provider reports both used and billable tokens, the instrumentation should report the billable count. When token counts can’t be obtained efficiently, the instrumentation should omit them rather than guess.两个值得了解的计数规则。当提供商同时报告已用和可计费 token 时，插桩应报告可计费数量。当无法高效获取 token 计数时，插桩应省略而非猜测。
These two metrics answer most operational questions: which model is most expensive, where latency is worst, how token consumption is trending. In our GenAI demo, they’re written to GreptimeDB via OTLP and queried with PromQL:这两个指标回答了大多数运维问题：哪个模型最贵、哪里延迟最差、token 消耗趋势如何。在我们的 GenAI 演示中，它们通过 OTLP 写入 GreptimeDB，并使用 PromQL 查询：
# p95 token consumption
histogram_quantile(0.95,
sum(rate(gen_ai_client_token_usage_bucket[5m])) by (le, gen_ai_token_type)
)Layer 6: Provider-Specific Conventions — From Generic to Specialized第六层：提供商特定约定 — 从通用到专用
Generic GenAI attributes cover common ground. Each provider has unique capabilities, handled through provider-specific conventions.通用 GenAI 属性覆盖了共同点。每个提供商都有独特能力，通过提供商特定约定处理。
OpenAIOpenAI
OpenAI has the most detailed provider convention so far[17]. With gen_ai.provider.name set to openai, the spec adds:OpenAI 拥有迄今为止最详细的提供商约定[17]。当 gen_ai.provider.name 设置为 openai 时，规范增加了：
gen_ai.usage.cache_read.input_tokens: tokens read from the provider's cachegen_ai.usage.cache_read.input_tokens：从提供商缓存读取的 tokengen_ai.usage.cache_creation.input_tokens: tokens written to the provider's cachegen_ai.usage.cache_creation.input_tokens：写入提供商缓存的 tokengen_ai.usage.reasoning.output_tokens: tokens consumed during reasoning (o1/o3 series, added in v1.41[8:4])gen_ai.usage.reasoning.output_tokens：推理过程中消耗的 token（o1/o3 系列，在 v1.41[8:4] 中新增）
Cached input is typically cheaper than regular input (exact discounts vary by model). Reasoning tokens are an additional cost specific to reasoning models.缓存输入通常比常规输入便宜（具体折扣因模型而异）。推理 token 是推理模型特有的额外成本。
Anthropic, AWS Bedrock, Azure AI InferenceAnthropic、AWS Bedrock、Azure AI Inference
Anthropic (gen_ai.provider.name=anthropic) includes a calculation guide for gen_ai.usage.input_tokens[7:1], because its billing model differs from OpenAI's. AWS Bedrock (aws.bedrock) and Azure AI Inference (azure.ai.inference) extend platform-specific attributes.Anthropic（gen_ai.provider.name=anthropic）包含 gen_ai.usage.input_tokens[7:1] 的计算指南，因为其计费模型与 OpenAI 不同。AWS Bedrock（aws.bedrock）和 Azure AI Inference（azure.ai.inference）扩展了平台特定属性。
Design principle: gen_ai.provider.name is the discriminator. It determines which provider-specific attributes should appear. An OpenAI span shouldn't carry aws.bedrock.* attributes, and vice versa.设计原则：gen_ai.provider.name 是鉴别器。它决定了哪些提供商特定属性应出现。OpenAI span 不应携带 aws.bedrock.* 属性，反之亦然。
A Complete Trace完整追踪
Combining all six layers, here’s what a full trace looks like when an agent calls an external tool via MCP:结合所有六个层次，以下是 Agent 通过 MCP 调用外部工具时的完整追踪示例：
invoke_agent support-router (INTERNAL, trace=t1)
│
├── chat gpt-4o (CLIENT)
│ gen_ai.provider.name = openai
│ gen_ai.request.model = gpt-4o
│ gen_ai.usage.input_tokens = 1523
│ gen_ai.usage.output_tokens = 42
│ gen_ai.response.finish_reasons = ["tool_calls"]
│
├── tools/call query-orders (CLIENT) ← MCP client
│ mcp.method.name = tools/call
│ mcp.session.id = sess-abc
│ gen_ai.tool.name = query-orders
│ │
│ └── tools/call query-orders (SERVER) ← MCP server
│
└── chat gpt-4o (CLIENT)
gen_ai.usage.input_tokens = 2841
gen_ai.usage.output_tokens = 256
gen_ai.usage.cache_read.input_tokens = 1523 ← OpenAI-specific
gen_ai.response.finish_reasons = ["stop"]
Metrics (same time window):
gen_ai.client.operation.duration{model=gpt-4o}
gen_ai.client.token.usage{model=gpt-4o, token_type=input}
mcp.client.operation.duration{method=tools/call}
Events (opt-in):
gen_ai.client.inference.operation.details → full prompt/completion
gen_ai.evaluation.result → score.value=0.92, score.label="relevant"Note: metrics use Prometheus-style shorthand ({model=...}). Actual attribute names are gen_ai.request.model, gen_ai.token.type, etc., exported to Prometheus as gen_ai_request_model, gen_ai_token_type. Prometheus 3.0 has since added native support for OpenTelemetry naming conventions, so cross-system alignment no longer requires manual conversion.注意：指标使用 Prometheus 风格简写（{model=...}）。实际属性名称为 gen_ai.request.model、gen_ai.token.type 等，导出到 Prometheus 后变为 gen_ai_request_model、gen_ai_token_type。Prometheus 3.0 已原生支持 OpenTelemetry 命名约定，因此跨系统对齐不再需要手动转换。
A single trace_id links the entire chain, from the agent's initial decision through the MCP server's execution to the final response.单个 trace_id 链接整个链，从 Agent 的初始决策到 MCP 服务器的执行再到最终响应。
Current Status and Adoption当前状态和采用情况
The spec is still in Development status. v1.36 is the transition baseline: existing instrumentations default to the old attribute format, while OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental switches to the latest version[2:2].规范仍处于开发状态。v1.36 是过渡基线：现有插桩默认使用旧属性格式，而 OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental 切换到最新版本[2:2]。
The OpenAI Python SDK instrumentation is the most mature[11:1]. Community libraries like OpenLLMetry[18] cover Anthropic, Cohere, and AWS Bedrock. Framework instrumentations for LangGraph and CrewAI are in progress.OpenAI Python SDK 插桩是最成熟的[11:1]。社区库如 OpenLLMetry[18] 覆盖了 Anthropic、Cohere 和 AWS Bedrock。LangGraph 和 CrewAI 的框架插桩正在进行中。
Datadog was one of the first commercial platforms to natively support v1.37+ GenAI semantic conventions[19]. Elastic’s 2026 observability report finds that 85% of organizations use some form of GenAI for observability, and 89% of OTel production users rate vendor compliance as “critical” or “very important”[20].Datadog 是最早原生支持 v1.37+ GenAI 语义约定的商业平台之一[19]。Elastic 的 2026 年可观测性报告发现，85% 的组织使用某种形式的 GenAI 进行可观测性，89% 的 OTel 生产用户将供应商合规性评为“关键”或“非常重要”[20]。
We built a full LLM observability stack on this spec in our GenAI demo, with OTLP going straight to GreptimeDB and all three signal types in unified storage:我们在 GenAI 演示中基于此规范构建了完整的 LLM 可观测性栈，OTLP 直接发送到 GreptimeDB，所有三种信号类型统一存储：
docker compose --profile load up -d
# Grafana: http://localhost:3000Wrap-Up总结
The six layers covered above (Client Spans, Agent Spans, MCP conventions, Events, Metrics, and Provider conventions) together span the full observability chain, from the initial model call to agent orchestration to tool execution.上述六个层次（客户端 Span、Agent Span、MCP 约定、事件、指标和提供商约定）共同覆盖了从初始模型调用到 Agent 编排再到工具执行的完整可观测性链。
The two pieces worth paying closest attention to today are Agent Spans and MCP conventions. Both are new to OTel, both target concrete problems (black-box agent reasoning and broken MCP traces), and both are likely to set the pattern for how the ecosystem instruments agents going forward. The three-mode design for content capture is the spec’s pragmatic answer to a real conflict between privacy and debuggability.今天最值得关注的两个部分是 Agent Span 和 MCP 约定。两者都是 OTel 的新内容，都针对具体问题（黑盒 Agent 推理和断裂的 MCP 追踪），并且很可能为生态系统未来如何插桩 Agent 设定模式。内容捕获的三种模式设计是规范对隐私与可调试性之间实际冲突的务实回答。
The spec itself is still moving fast. Every release from v1.37 to v1.41 has touched GenAI. For teams building LLM or agent systems, the easiest place to start is the OpenAI Python SDK instrumentation. From there, the spec docs and our demo show how the layers fit together in practice.规范本身仍在快速发展。从 v1.37 到 v1.41 的每个版本都涉及 GenAI。对于构建 LLM 或 Agent 系统的团队，最简单的起点是 OpenAI Python SDK 插桩。从那里，规范文档和我们的演示展示了各层在实际中如何组合。
References参考文献
- OpenTelemetry GenAI SIG ↩︎OpenTelemetry GenAI SIG ↩︎
- Semantic Conventions for Generative AI Systems (docs v1.41.0, Development) ↩︎ ↩︎ ↩︎生成式 AI 系统的语义约定（文档 v1.41.0，开发中） ↩︎ ↩︎ ↩︎
- Semantic Conventions v1.41.1 ↩︎语义约定 v1.41.1 ↩︎
- v1.37.0 Release ↩︎ ↩︎v1.37.0 发布 ↩︎ ↩︎
- v1.38.0 Release ↩︎v1.38.0 发布 ↩︎
- v1.39.0 Release ↩︎ ↩︎v1.39.0 发布 ↩︎ ↩︎
- v1.40.0 Release ↩︎ ↩︎v1.40.0 发布 ↩︎ ↩︎
- v1.41.0 Release ↩︎ ↩︎ ↩︎ ↩︎ ↩︎v1.41.0 发布 ↩︎ ↩︎ ↩︎ ↩︎ ↩︎
- Semantic Conventions 2026 Roadmap ↩︎语义约定 2026 路线图 ↩︎
- GenAI Client Spans ↩︎ ↩︎ ↩︎GenAI 客户端 Span ↩︎ ↩︎ ↩︎
- opentelemetry-instrumentation-openai-v2 ↩︎ ↩︎opentelemetry-instrumentation-openai-v2 ↩︎ ↩︎
- GenAI Agent and Framework Spans ↩︎GenAI Agent 和框架 Span ↩︎
- Glama: OpenTelemetry for MCP Analytics and Agent Observability ↩︎Glama：用于 MCP 分析和 Agent 可观测性的 OpenTelemetry ↩︎
- MCP Semantic Conventions ↩︎ ↩︎ ↩︎MCP 语义约定 ↩︎ ↩︎ ↩︎
- GenAI Events ↩︎ ↩︎ ↩︎GenAI 事件 ↩︎ ↩︎ ↩︎
- GenAI Metrics ↩︎GenAI 指标 ↩︎
- OpenAI Semantic Conventions ↩︎OpenAI 语义约定 ↩︎
- OpenLLMetry project ↩︎OpenLLMetry 项目 ↩︎
- Datadog: LLM OTel Semantic Convention ↩︎Datadog：LLM OTel 语义约定 ↩︎
- Elastic: Observability trends for 2026 ↩︎Elastic：2026 年可观测性趋势 ↩︎
