Reilly WoodReilly Wood
I work on Datadog’s official MCP (Model Context Protocol) server, our first observability interface designed specifically for customers’ AI agents. Our first version was a thin wrapper around existing APIs, the kind of thing you can build in a weekend. It worked well enough to validate the idea, but then we started watching agents actually use it to solve real problems.我在 Datadog 官方的 MCP（模型上下文协议）服务器上工作，这是我们为客户的 AI 代理专门设计的首个可观测性接口。我们的第一个版本只是对现有 API 的薄包装，类似于周末就能搭建的东西。它足以验证概念，但随后我们看到代理真正使用它来解决实际问题。
Agents would fill their context windows with log data and lose track of what they were doing. They’d request what seemed like a reasonable number of records, then blow their token budget because a few of those records happened to be huge. They’d try to answer questions about trends by retrieving raw samples and guessing. We found out quickly that “just expose your APIs” wasn’t going to cut it.代理会用日志数据填满它们的上下文窗口，导致失去对正在做的事情的跟踪。它们会请求看似合理数量的记录，却因为其中几条记录异常庞大而耗尽 token 预算。它们会尝试通过检索原始样本并猜测来回答趋势问题。我们很快发现，仅仅“暴露你的 API”根本不够。
We’ve since rethought almost everything about our tool design to make observability actually ergonomic for agents. In this post, I’ll share some of what we learned from that process.此后我们几乎重新思考了工具设计的所有方面，以让可观测性真正对代理友好。在本文中，我将分享我们从这一过程学到的一些经验。
Context windows fill up fast上下文窗口很快就会被占满
The first thing we learned is that context efficiency matters a lot. When an agent calls a tool, the entire result ends up in the context window, and observability data can be large. Our early tools would sometimes return thousands of log records, each with dozens of fields, and agents would choke on the results.我们首先了解到上下文效率非常重要。当代理调用工具时，整个结果都会进入上下文窗口，而可观测性数据可能非常庞大。我们早期的工具有时会返回数千条日志记录，每条记录包含数十个字段，导致代理无法处理这些结果。
We attacked this from a few angles.我们从几个角度着手解决。
Format matters. Our first prototype just returned JSON from our APIs, which is fine for programmatic consumption but often wasteful for agents. Take this:格式很重要。我们的第一个原型直接返回 API 的 JSON，这对程序化消费来说没问题，但对代理来说往往浪费空间。来看这个例子：
[ { "firstName": "Alice", "lastName": "Johnson", "age": 28 }, { "firstName": "Bob", "lastName": "Smith", "age": 35 }]Compare to the same data in CSV:与 CSV 中相同的数据进行比较：
firstName,lastName,ageAlice,Johnson,28Bob,Smith,35CSV uses about half as many tokens per record (the exact number varies by tokenizer). For tabular data without nesting, CSV or TSV is almost always the right choice. For nested data, YAML is a good middle ground—you can usually shave around 20% off token count just by switching from JSON to YAML. If you’re familiar with TOON, this is the core insight behind it.CSV 每条记录使用的 token 大约只有一半（具体数量取决于分词器）。对于没有嵌套的表格数据，CSV 或 TSV 几乎总是最佳选择。对于嵌套数据，YAML 是一个不错的折中——仅通过从 JSON 切换到 YAML 就能削减约 20% 的 token 数。如果你熟悉 TOON，这正是其核心洞见。
Trim what you don’t need. Our log records had dozens of fields, but agents rarely needed all of them. By trimming rarely used fields from the default output—and letting agents request them back if needed—we saw another big improvement.去除不必要的字段。我们的日志记录有数十个字段，但代理很少需要全部。通过在默认输出中去除不常用的字段——并在需要时让代理自行请求——我们又获得了显著提升。
The cumulative effect of these two changes was significant. For some tools, we can now fit about 5x more records in the same number of tokens. This makes a huge difference when agents are digging through large volumes of data.这两项改动的累计效果相当可观。对于某些工具，我们现在可以在相同 token 数量下容纳约 5 倍的记录。这在代理需要挖掘大量数据时意义巨大。
Rethink your approach to pagination. APIs are typically paginated by record count, but records can vary widely in size. In our case, a Datadog log message might be only 100 characters or it might be 1 MB. We ran into situations where an agent would request a reasonable number of logs and then the context window was gone because a few of the logs were huge. So we switched to paginating by token budget: The server cuts off its response after a certain number of tokens and returns a cursor for more.重新思考分页方式。API 通常按记录数分页，但记录大小差异很大。以我们为例，Datadog 的一条日志可能只有 100 字符，也可能高达 1 MB。我们遇到过代理请求合理数量的日志，却因为几条巨大的日志导致上下文窗口瞬间耗尽的情况。因此我们改为按 token 预算分页：服务器在达到一定 token 数后截断响应，并返回用于获取更多内容的游标。
That said, some of this may matter less in the future. Tools like Cursor and Claude Code now write long tool results to disk instead of putting everything in context. This isn’t in the MCP spec yet, but if it becomes widespread, format efficiency will be less critical.不过，这些在未来可能不那么重要。Cursor 和 Claude Code 等工具现在会把长工具结果写入磁盘，而不是全部放入上下文。虽然这尚未写入 MCP 规范，但如果被广泛采用，格式效率的重要性将会下降。
Let agents query, not just retrieve让代理进行查询，而不仅仅是检索
Agents often need to do more than retrieve raw data, so efficient formats can only get you so far.代理往往需要的不止是原始数据的检索，仅靠高效格式也只能帮到有限程度。
For example, a user might ask their agent, “Which services are logging the most errors in the last hour?” Our first logs tools could only retrieve logs that matched filter criteria, so agents would attempt to answer this question by pulling some logs and inferring trends from that sample. This was wasteful and often incorrect. Even worse, some agents would try to brute-force it, repeatedly retrieving data until the context window filled up.例如，用户可能会问他们的代理：“过去一小时哪些服务记录的错误最多？”我们最早的日志工具只能检索符合过滤条件的日志，导致代理只能通过拉取一些日志并从样本中推断趋势来回答，这既浪费又常常不准确。更糟的是，有些代理会尝试暴力检索，反复获取数据直至上下文窗口被填满。
I’m a databases guy at heart, and I quickly realized that this is exactly the kind of problem I would try to solve with SQL—so why not let agents do the same? Instead of retrieving raw logs, agents can now write a query like:我本质上是个数据库人，我很快意识到这正是我会用 SQL 解决的问题——那为何不让代理也这么做呢？于是我们让代理可以写类似下面的查询：
SELECT service, COUNT(*) as error_countFROM logsWHERE status = 'error'GROUP BY serviceORDER BY error_count DESCLIMIT 10This gives the right answer quickly, in very few tokens. SQL has worked really well for us: Agents are quite good at writing it, and it gives them fine-grained control over what data ends up in their context window. Supporting this was a significant lift—at the scale we operate at, traditional relational databases don’t work—but it’s been worth it.这能快速给出正确答案，且消耗的 token 极少。SQL 对我们帮助很大：代理擅长编写它，并且它让代理能够细粒度控制哪些数据进入上下文窗口。实现这点投入不小——在我们规模下，传统关系型数据库并不适用——但值得。
One pleasant surprise was that SQL tools didn’t just improve correctness; they also reduced costs for users. In some of our eval scenarios, runs were about 40% cheaper because agents used fewer tokens to reach answers. They could SELECT only the fields they needed, LIMIT results to a few rows, or count efficiently instead of retrieving large volumes of raw data.一个令人惊喜的发现是，SQL 工具不仅提升了正确性，还降低了用户成本。在一些评估场景中，运行成本下降约 40%，因为代理使用更少的 token 就能得到答案。它们可以只 SELECT 所需字段、LIMIT 结果行数，或高效计数，而无需检索大量原始数据。
Tools aren’t free工具不是免费的
A “just turn every API into a tool” approach doesn’t scale. As the number of tools grows, agents struggle with accurate tool calling, and each tool’s description takes up context window space. We also need to be mindful that Datadog might not be the only MCP server connected to an agent, which makes being frugal with context even more important.“把每个 API 都变成工具”的做法无法扩展。随着工具数量增长，代理在准确调用工具方面会遇到困难，而且每个工具的描述也会占用上下文窗口空间。我们还必须注意，Datadog 可能不是唯一连接到代理的 MCP 服务器，这使得上下文的节约更加重要。
We’ve tried a few approaches to keep tool count down:我们尝试了几种方式来控制工具数量：
Flexible tools: Rather than one tool per API endpoint, we design tools that can serve multiple use cases. This requires careful schema design, but one well-designed tool can often do the work of several narrow ones.灵活的工具：我们不为每个 API 端点单独设计工具，而是设计能够服务多种用例的工具。这需要精心的模式设计，但一个设计良好的工具往往可以替代多个狭窄的工具。
Toolsets: By default, connecting to our MCP server gives users a core set of tools for common workflows. Datadog is a large platform, though, so we also support opt-in toolsets for more specialized needs. The downside is that users need to anticipate what capabilities their agent will require ahead of time.工具集：默认情况下，连接到我们的 MCP 服务器会为用户提供一套核心工具，以满足常见工作流。Datadog 平台很大，所以我们也支持可选的工具集，以满足更专业的需求。缺点是用户需要提前预估代理将需要哪些能力。
Layering: We explored patterns where agents chain tool calls—one tool to ask “how do I accomplish X?” and another to actually do it. Block has written a great blog post about this approach. The advantage is that you can expose specialized functionality without cramming it all into the context window up front. The tradeoff is latency: a task that once required one tool call now requires two, which can noticeably slow down agent sessions.分层调用：我们探索了代理链式调用工具的模式——一个工具询问“如何实现 X？”，另一个实际执行。Block 写了一篇很棒的博客文章阐述了这种方法。优势在于可以在不一次性把所有功能塞进上下文的情况下暴露专用功能。代价是延迟：原本一次调用的任务现在需要两次调用，可能会明显拖慢代理会话。
Over time, these approaches may become less necessary. Agents are getting smarter at managing their own context. Tools like Claude Code now use tool search to avoid loading every tool up front, and skills let agents load specialized knowledge on demand. Exactly how skills and MCP fit together is still an open question (we’re trying out approaches like Kiro Powers), but we’re excited to do more with skills in the future.随着时间推移，这些方法可能会变得不那么必要。代理在管理自身上下文方面越来越聪明。Claude Code 等工具现在使用工具搜索来避免一次性加载所有工具，技能（skills）让代理按需加载专用知识。技能与 MCP 如何协同仍是未解之谜（我们正在尝试 Kiro Powers 等方案），但我们对未来在技能方面的投入充满期待。
Guide the agent引导代理
Early on, we saw agents fail in ways that were hard to diagnose. They’d send a malformed query, get back a generic error, and then try the exact same thing again. Or give up entirely. It took us a while to realize that the problem wasn’t the agents—it was us.早期我们看到代理出现难以诊断的失败。它们会发送格式错误的查询，收到通用错误后再次尝试相同操作，甚至直接放弃。我们花了一段时间才意识到问题不在代理，而在我们这边。
Error messages matter more than you’d think. Agents are surprisingly good at recovering from errors, but they need specifics. An error message like “invalid query” usually isn’t helpful; something like “unknown field ‘stauts’ – did you mean ‘status’?” gives the agent a clear next step. We put a lot of effort in making our error messages specific and actionable, and it paid off.错误信息比想象中更重要。代理在错误恢复方面出奇地好，但需要具体信息。像“invalid query”这样的错误信息通常没有帮助；而“unknown field ‘stauts’ – did you mean ‘status’?” 能为代理提供明确的下一步。我们投入大量精力让错误信息具体且可操作，收到了成效。
Make documentation discoverable. We have a search_datadog_docs tool that does a RAG-powered search over Datadog’s documentation. We encourage agents to use it (via server instructions) when they’re unsure about query syntax or available options. This lets us avoid cramming every detail into tool descriptions, while still giving agents a way to look things up on demand.让文档易于发现。我们有一个 search_datadog_docs 工具，可对 Datadog 文档进行 RAG 驱动的搜索。我们鼓励代理在不确定查询语法或可用选项时使用它（通过服务器指令），这样既避免把所有细节塞进工具描述，又能让代理按需查找信息。
Tool results can include guidance, not just data. This is quite a departure from traditional REST API design, where there’s usually nothing that can reason about plain text calling your API. Our logs tools still return exactly what the agent requested, but sometimes we’ll add a short note like, “You searched for the payment service, did you mean to search for the payments service instead?”工具结果可以包含指导，而不仅仅是数据。这与传统的 REST API 设计大相径庭，后者通常无法对纯文本调用进行推理。我们的日志工具仍然返回代理请求的内容，但有时会附加简短提示，例如：“您搜索的是 payment service，是否想搜索 payments service？”
Specialized vs. general-purpose专用 vs 通用
Datadog also has Bits AI SRE, a hosted agent that investigates alerts and suggests remediations. It differs from agents using the MCP server in that it has a web UI and is purpose-built for that specific workflow.Datadog 还有 Bits AI SRE，这是一个托管代理，负责调查告警并提供修复建议。它与使用 MCP 服务器的代理不同，因为它拥有 Web UI，且专为该工作流构建。
There’s a real tradeoff here. Bits AI SRE can make assumptions that a general MCP server can’t: It knows the user is investigating an alert, so it can proactively pull in related data and offer specialized tools and UI for that use case. An MCP server has to be more general—it needs to work across many workflows without making strong assumptions up front.这里存在真实的权衡。Bits AI SRE 可以做出通用 MCP 服务器做不到的假设：它知道用户正在调查告警，因此可以主动拉取相关数据并提供专用工具和 UI。MCP 服务器必须更通用——需要在不做强假设的前提下支持多种工作流。
I don’t think one approach will win out. Specialized agents will probably always have an edge for well-defined workflows, but MCP offers flexibility—you can plug Datadog into Claude Code, a homegrown agent, or whatever comes next. We’re working on bringing these closer together, exposing Bits AI SRE’s capabilities through MCP and making the specialized agent more flexible about what it can investigate. Over time, the line between “specialized agent” and “MCP server with good defaults” may get blurry.我不认为单一方案会占据全部市场。专用代理在明确工作流中始终会有优势，但 MCP 提供了灵活性——你可以把 Datadog 接入 Claude Code、自研代理或其他未来系统。我们正在让两者更紧密结合，通过 MCP 暴露 Bits AI SRE 的能力，并让专用代理在可调查范围上更灵活。随着时间推移，“专用代理”与“具备良好默认值的 MCP 服务器”之间的界限可能会变得模糊。
The takeaways要点回顾
There are no textbooks for building MCP servers yet. Most of what we’ve learned has come from working closely with customers and watching agents fail in real scenarios, then trying to figure out why.目前还没有关于构建 MCP 服务器的教材。我们的大部分经验来自与客户的紧密合作以及观察代理在真实场景中的失败，然后分析原因。
If I had to boil that experience down to a few principles, they’d look like this:如果要把这些经验浓缩为几条原则，大致如下：
Don’t just wrap your APIs. Design tools around agents’ constraints.不要只包装你的 API。围绕代理的约束来设计工具。
Be frugal with context windows, and give agents the tools to be frugal, too. (Query languages help.)对上下文窗口要节约，并为代理提供同样节约的工具。（查询语言有帮助。）
Guide agents with good error messages and discoverable documentation.用良好的错误信息和易发现的文档来引导代理。
This space is moving fast, and some of today’s constraints may relax over time. But you don’t have to wait for that. These lessons are what helped us ship a useful MCP server today, and they’re shaping how we think about agent-facing tools at Datadog going forward.这个领域发展迅速，今天的一些限制未来可能会放宽。但你不必等到那时。这些经验帮助我们今天交付了实用的 MCP 服务器，也在塑造 Datadog 对面向代理工具的未来思考。
Interested in building agent-friendly systems at scale? We’re hiring! 想要在大规模构建友好的代理系统吗？我们正在招聘！
