Your LLM Has a Spending Problem - How Prompts, Context Windows, and Agent Loops Quietly Torch Your Budget您的LLM存在支出问题——提示词、上下文窗口和代理循环如何悄悄烧掉您的预算

Everybody loves AI until the invoice shows up!人人都爱AI，直到账单出现！

Not the cute "we spent a few hundred dollars experimenting" invoice that gets hand-waved away as innovation spend. I mean the kind of invoice that makes finance suddenly become deeply curious about architecture decisions they previously considered engineering trivia, while operations and/or product teams insist everything is still technically a pilot, and leadership starts asking whether we can "optimize this" as though that thought had just occurred to humanity.不是那种可爱的“我们花了几百美元试验”的账单，那种会被当作创新支出而一笔勾销。我指的是那种让财务部门突然对之前视为工程琐事的架构决策产生浓厚兴趣，而运营和/或产品团队仍坚称一切还是试点，领导层开始询问能否“优化它”——仿佛这个想法刚被人类想到似的。

If you have been around enterprise technology for any length of time, this probably feels familiar, because we did this exact dance with cloud. We got excited about elasticity, speed, self-service, digital transformation, and developer empowerment. Then we discovered that moving fast without an operational model produces some incredibly educational invoices. FinOps emerged because cloud economics got weird enough that traditional budgeting and governance models stopped being sufficient.如果你在企业技术领域待过一段时间，这大概会让你感到熟悉，因为我们与云也跳过完全相同的舞步。我们曾为弹性、速度、自助服务、数字化转型和开发者赋能而兴奋。然后我们发现，在没有运营模型的情况下快速推进会产生一些极具教育意义的账单。FinOps之所以出现，是因为云经济变得足够奇怪，以至于传统的预算和治理模型已不再足够。

AI is following the same path, except the economics are stranger, because the thing driving spend is not always infrastructure. Sometimes it is architecture. Sometimes it is workflow design. Sometimes it is application behavior. And sometimes it is a prompt that grew into an ungoverned monster because everyone kept adding "just one more helpful instruction."AI正沿着同样的道路前进，只不过经济性更为奇怪，因为推动支出的并不总是基础设施。有时是架构，有时是工作流设计，有时是应用行为。有时则是一个因为每个人都不断添加“再多一条有用的指令”而成长为无法控制的怪物的提示词。

That last one happens more than people would like to admit.最后一种情况发生的频率比人们愿意承认的要高。

AI Cost Is Not "Just Another Cloud Bill"AI成本并非“另一张云账单”

One of the biggest mistakes I see right now is treating AI cost management like it is cloud FinOps with slightly different line items. That instinct makes sense, because most of us already know how to reason through cloud spend. We understand compute utilization, storage growth, idle resources, reserved pricing, licensing efficiency, network egress, and the thousand small architectural decisions that quietly evolve into larger financial problems when nobody is paying attention.我目前看到的最大的错误之一，就是将AI成本管理视为略有不同条目的云FinOps。这种直觉有道理，因为大多数人已经知道如何分析云支出。我们了解计算利用率、存储增长、闲置资源、预留定价、许可效率、网络出站流量，以及那些在无人注意时悄然演变成更大财务问题的无数小架构决策。

Those instincts still matter, but AI changes the shape of the problem, because cost is often driven by interaction behavior rather than static infrastructure allocation. A virtual machine is fairly honest about what it is doing. Storage accounts are not emotionally complicated (at least not yet). Network egress may occasionally feel personally insulting, but at least it behaves predictably. AI workloads can be much more chaotic, because tiny implementation decisions dramatically change cost behavior. A prompt gets longer. A context window expands. A retrieval pipeline injects more content. An agent retries failed workflows. A product team decides the assistant should feel "more conversational." Suddenly the conversation is no longer about infrastructure optimization. It is about behavioral economics hiding inside application design.这些直觉仍然重要，但AI改变了问题的形态，因为成本往往由交互行为驱动，而非静态基础设施分配。虚拟机对自己在做什么相当诚实。存储账户没有情感上的复杂性（至少目前没有）。网络出站流量有时可能让人感觉受到侮辱，但至少它行为可预测。AI工作负载可能混乱得多，因为微小的实现决策会极大地改变成本行为。提示词变长；上下文窗口扩大；检索管道注入更多内容；代理重试失败的工作流；产品团队决定助手应该“更健谈”。突然之间，讨论不再关于基础设施优化，而是关于隐藏在应用设计中的行为经济学。

That is a very different operational challenge, and if you approach it with only traditional infrastructure instincts, you are going to miss some very expensive patterns.这是一个非常不同的运营挑战，如果你仅凭传统基础设施直觉来应对，就会错过一些非常昂贵的模式。

Tokens Are Tiny Budget GremlinsToken是预算小精灵

Let us talk about tokens, because AI conversations have a habit of casually using the term like everyone instinctively understands what it means, when in reality half the room is only quietly nodding while internally wondering if they should have asked sooner. Technically, tokens are chunks of text processed by the model. They are not exactly words and not exactly characters, and the exact implementation varies by provider. That distinction matters for precision, but from a practical FinOps perspective the important thing is much simpler: tokens are how you get billed! What you send into the model consumes tokens. What comes back out consumes tokens. Now, depending on the provider, cached interactions, tool execution, and orchestration overhead can create additional economic nuance which can turn simple assumptions into dangerously incomplete ones. It's...a lot...and confusing.我们来谈谈token，因为AI对话习惯性地随意使用这个词，好像大家都本能地理解它的含义，而实际上房间里一半人只是在默默点头，内心却在想是否应该早点问清楚。从技术上讲，token是由模型处理的文本片段。它们不完全是单词，也不完全是字符，具体实现因提供商而异。这种区别对于精确性很重要，但从实际FinOps的角度来看，重要的事情要简单得多：token就是你被收费的方式！你发送给模型的内容消耗token，模型返回的内容也消耗token。此外，根据提供商不同，缓存的交互、工具执行和编排开销可能会产生额外的经济细微差别，使简单的假设变得危险地不完整。这……很多……而且令人困惑。

This is where AI spending starts behaving in ways traditional infrastructure teams find deeply irritating. A short request asking for a summary may barely register. A heavily engineered enterprise prompt with role instructions, governance controls, formatting directives, prior conversation history, retrieved documents, workflow metadata, and supporting tool definitions is a very different story. Then the model responds with a wonderfully verbose answer, because apparently we have all unintentionally reinforced the belief that concise answers are somehow less intelligent.这正是AI支出开始以传统基础设施团队深感恼火的方式表现的地方。一个要求摘要的简短请求可能几乎不产生什么影响。一个精心设计的企业级提示词，包含角色指令、治理控制、格式指令、先前对话历史、检索文档、工作流元数据和辅助工具定义，则是另一回事。然后模型用一个非常冗长的答案回应，因为显然我们都不经意间强化了这样一个信念：简洁的答案在某种程度上不够智能。

The outcome is you are now paying for input, output, and potentially a collection of supporting behaviors that completely change the economics of what looked, at first glance, like a simple interaction. A poorly sized VM does not become dramatically more expensive because someone reworded a sentence, whereas an LLM absolutely can.结果是，你现在为输入、输出以及可能一系列辅助行为付费，这些行为完全改变了一开始看似简单交互的经济性。一个大小不当的虚拟机不会因为有人改写了一个句子而变得极其昂贵，而LLM绝对可以。

Your Architecture Has an AI Cost Personality你的架构有AI成本个性

If you have followed my FinOps content before, you know I like talking about architecture having a cost personality, because different design decisions create different economic behaviors. AI takes that concept and cranks the intensity levers.如果你之前关注过我的FinOps内容，你知道我喜欢谈论架构具有成本个性，因为不同的设计决策会产生不同的经济行为。AI将这个概念推向了极致。

A lightweight FAQ chatbot has one personality. A retrieval-augmented knowledge assistant has another. A proposal analysis workflow behaves differently than a coding copilot. An internal enterprise knowledge assistant behaves differently than a customer-facing support implementation. A multi-agent orchestration model, where specialized agents hand work between each other, may be technically elegant and completely appropriate for the business use case, but it does not behave economically like a straightforward prompt-response implementation.轻量级FAQ聊天机器人有一种个性。检索增强的知识助手有另一种个性。提案分析工作流的行为不同于编码助手。内部企业知识助手的行为不同于面向客户的支持实现。多代理编排模型，其中专门代理之间交接工作，可能在技术上优雅并且完全适合业务用例，但它在经济上的行为并不像直接的提示-响应实现那样。

That matters, because I still hear people talk about "AI spend" like it is a meaningful single category. Know it's not. To me, that's like talking about "cloud spend" without distinguishing between ephemeral development workloads, enterprise ERP infrastructure, serverless event processing, and data lake analytics. The architecture underneath determines the economics, and pretending all AI workloads behave similarly leads to terrible optimization decisions.这很重要，因为我仍然听到人们谈论“AI支出”，仿佛它是一个有意义的单一类别。要知道并非如此。对我来说，这就像谈论“云支出”而不区分临时开发工作负载、企业ERP基础设施、无服务器事件处理和数据湖分析。底层的架构决定了经济性，假装所有AI工作负载行为相似会导致糟糕的优化决策。

If you're building a simple assistant answering short transactional questions, your economic expectations should look very different than if you're building an agentic workflow that reasons, retrieves information, invokes tools, validates outcomes, retries failures, and refines results before responding. If you're embedding copilots into employee productivity workflows, your cost model should look different than a customer-facing conversational experience handling thousands of interactions.如果你在构建一个回答简短交易问题的简单助手，你的经济预期应该与构建一个代理工作流截然不同，后者会推理、检索信息、调用工具、验证结果、重试失败并在回应前完善结果。如果你将助手嵌入员工生产力工作流，你的成本模型应该与处理数千次交互的面向客户的对话体验不同。

Architecture shapes spend and always has...AI just makes that relationship noisier.架构塑造支出，一直以来都是如此……AI只是让这种关系更加嘈杂。

Context Windows: Where Good Intentions Go Shopping上下文窗口：良好意图的挥霍之地

One of the easiest ways teams accidentally build expensive AI systems is through context accumulation, because every contributing decision sounds perfectly reasonable in isolation. You want the assistant to feel conversational, so you preserve chat history. You want better answers, so you implement retrieval-augmented generation and inject supporting documentation. You want stronger governance, so you add system instructions around acceptable use, formatting, tone, and policy controls. You want personalization, so user context gets appended. You want tooling, so orchestration metadata and function definitions are included. Product wants a smarter experience. Security wants stronger guardrails. Ultimately everyone wants their enhancement to make the system better. This has all the makings of what I like to call enterprise architecture drama. Taken individually, these are not reckless decisions. Collectively, they can (and often) get hilariously expensive.团队无意中构建昂贵AI系统的最简单方式之一是通过上下文累积，因为每个贡献决策孤立来看都完全合理。你希望助手显得健谈，所以保留聊天历史。你想要更好的答案，所以实现检索增强生成并注入支持文档。你想要更强的治理，所以添加关于可接受使用、格式、语气和政策控制的系统指令。你想要个性化，所以附加用户上下文。你想要工具支持，所以包含编排元数据和函数定义。产品想要更智能的体验。安全想要更强的护栏。最终，每个人都希望自己的增强能让系统更好。这具备了我称之为企业架构戏剧的所有要素。单独来看，这些都不是鲁莽的决定。但合在一起，它们可能（而且常常）变得离谱地昂贵。

A user asks something simple like "what changed in this contract draft?" and what actually gets sent to the model includes system instructions, governance rules, previous conversation history, retrieved content, metadata, tool definitions, formatting expectations, workflow state, and the actual question hanging out near the bottom wondering how its life got so complicated. Every time that request runs, you pay for the whole payload, not just the visible question.用户问一个简单的问题，比如“这份合同草案有什么变化？”，而实际发送给模型的内容包括系统指令、治理规则、先前的对话历史、检索到的内容、元数据、工具定义、格式期望、工作流状态，以及实际的问题——它挂在底部，纳闷自己的生命为何变得如此复杂。每次该请求运行时，你都要为整个负载付费，而不仅仅是可见的问题。

That is what makes this category of waste so sneaky. Nobody believes they are making irresponsible architectural decisions. They are making incremental product and operations improvements. The problem is that incremental improvements accumulate, and context windows are remarkably indifferent to your intentions.这就是这类浪费如此隐蔽的原因。没有人认为自己做出了不负责任的架构决策。他们只是在做渐进的产品和运营改进。问题在于，渐进式改进会累积，而上下文窗口对你的意图毫不在意。

Agents: Tiny Distributed Systems Wearing Friendly Faces代理：戴着友好面孔的微型分布式系统

This is where things get especially entertaining, so pull out the popcorn!这就是事情变得特别有趣的地方，所以拿出爆米花吧！

A straightforward chatbot interaction is easy to reason about. A user sends a request, the model processes request, and a response comes back. Costs scale in a reasonably understandable way.直接的聊天机器人交互很容易理解。用户发送请求，模型处理请求，然后返回响应。成本以合理可理解的方式扩展。

Agents are fundamentally different, because they transform what looks like a simple interaction into a potentially complex backend workflow. Instead of simply answering, the system may reason through the request, decide which tools to invoke, query external systems, validate outputs, retry failed actions, refine conclusions, and then finally produce a response. That entire exchange can be incredibly useful, but it can also become financially spectacular in all the wrong ways if no one is paying attention.代理从根本上来说不同，因为它们将一个看似简单的交互转化为可能复杂的后端工作流。系统不是简单地回答，而是可能对请求进行推理、决定调用哪些工具、查询外部系统、验证输出、重试失败的操作、完善结论，然后最终产生响应。整个交换可能非常有用，但如果没人注意，它也可能会在财务上变得惊人地糟糕。

From the user's perspective, they asked one question. From the backend's perspective, that request may have triggered multiple reasoning passes, tool calls, data lookups, validation logic, retries, re-evaluations, and formatting steps. That is not one interaction. That is a tiny, distributed workflow wearing a friendly conversational interface and quietly helping itself to your budget.从用户的角度来看，他们问了一个问题。从后端的角度来看，该请求可能触发了多次推理、工具调用、数据查找、验证逻辑、重试、重新评估和格式化步骤。这不是一次交互，而是一个微型的分布式工作流，戴着友好的对话界面，悄悄地从你的预算中取用。

To be clear, this is not Shannon's argument AGAINST agents. Agentic architectures absolutely have legitimate business value. This is an argument for understanding their economics. If you assume one user interaction equals one billable event and your architecture actually behaves like a multi-step autonomous workflow, your financial assumptions are already wrong.需要明确的是，这并不是Shannon反对代理的观点。代理架构绝对具有合理的商业价值。这是为了理解它们的经济性。如果你假设一次用户交互等于一个计费事件，而你的架构实际行为像一个多步骤自主工作流，那么你的财务假设已经错了。

Here's my rule of thumb: Autonomy is powerful. Autonomy is also expensive when left unsupervised.这是我的经验法则：自主性强大，但无人监督时也昂贵。

Traditional FinOps vs AI FinOps传统FinOps与AI FinOps

Traditional FinOps asks infrastructure-centric questions. Which workloads are expensive? Which resources are underutilized? Can we right size? Should we shift to reserved pricing? Are we retaining too much storage? Can licensing be optimized? These are useful questions, but AI introduces a different category of investigation, because inefficiencies often hide inside application behavior rather than obvious infrastructure waste.传统FinOps提出以基础设施为中心的问题。哪些工作负载昂贵？哪些资源未充分利用？我们能合理调整规模吗？是否应该转向预留定价？我们是否保留了太多存储？许可可以优化吗？这些都是有用的问题，但AI引入了不同类别的调查，因为效率低下通常隐藏在应用行为中，而不是明显的基础设施浪费中。

Now the questions get more interesting. Why is one workflow dramatically more expensive than another? Why is a simple assistant generating enough output to qualify as serialized fiction? Why are prompt payloads growing over time? Why is an agent retrying failed operations with the confidence of someone who has never encountered diminishing returns? Why is one business unit consuming dramatically more tokens than another for what appears to be the same use case?现在问题变得更有趣了。为什么一个工作流比另一个贵得多？为什么一个简单的助手产生的输出足以成为连载小说？为什么提示负载随时间增长？为什么代理以从未遇到过收益递减的人的信心重试失败的操作？为什么一个业务单元消耗的token比另一个多得多，而用例似乎相同？

These are not classic infrastructure questions, rather they are application behavior questions. That distinction matters, because traditional FinOps gives you a strong foundation but not a complete operating model for AI economics.这些不是经典的基础设施问题，而是应用行为问题。这种区别很重要，因为传统FinOps为你提供了坚实的基础，但并非AI经济的完整运营模型。

The optimization target should be shifting from infrastructure efficiency toward interaction efficiency.优化目标应从基础设施效率转向交互效率。

Unit Economics Are the Adult Conversation单位经济学是成人对话

This is the part executives usually understand immediately once the conversation is framed correctly. Nobody actually cares about AI spend in isolation. Rather, they care whether the spend creates value.这是高管通常能立即理解的部分，只要对话框架正确。实际上，没有人关心孤立的AI支出。他们关心的是支出是否创造价值。

A $20,000 monthly AI bill might be entirely justified if it meaningfully reduces support costs, accelerates engineering productivity, shortens proposal generation cycles, or improves customer outcomes. A much smaller bill may be wasteful if it powers something nobody meaningfully uses.每月2万美元的AI账单完全可能是合理的，如果它能显著降低支持成本、加速工程生产力、缩短提案生成周期或改善客户成果。一个小得多的账单可能浪费，如果它驱动的是没人实际使用的东西。

That means mature AI FinOps conversations need to move quickly toward unit economics:这意味着成熟的AI FinOps对话需要迅速转向单位经济学：

• What is the cost per support interaction?每次支持交互的成本是多少？

• What is the cost per contract review?每次合同审查的成本是多少？

• What is the cost per proposal generated?每次生成的提案的成本是多少？

• What is the cost per engineering productivity assist?每次工程生产力辅助的成本是多少？

• What is the cost per knowledge retrieval?每次知识检索的成本是多少？

• What is the cost per customer workflow completed?每次客户工作流完成的成本是多少？

Those are useful business conversations. "AI is expensive" is usually just budget anxiety masquerading as strategy. Budget anxiety usually also kills any and maybe all innovation.这些是有用的业务对话。“AI太贵”通常只是预算焦虑伪装成策略。预算焦虑往往也会扼杀任何，甚至所有创新。

Making This Operational: Provider Dashboards Help...使其可操作：提供商仪表板有帮助……

...but they are not governance.……但它们不是治理。

Provider-native telemetry matters, and you absolutely need it to tell the tale appropriately. OpenAI exposes usage reporting. Azure OpenAI provides monitoring telemetry through Azure observability tooling. Anthropic exposes detailed usage data through SDK responses, including the prompt caching fields that change the economics in ways naive counting misses. Google returns usage metadata on every Gemini response. That is useful operational visibility, but it is not governance.提供商原生的遥测很重要，你绝对需要它来恰当地讲述故事。OpenAI公开使用情况报告。Azure OpenAI通过Azure可观测性工具提供监控遥测。Anthropic通过SDK响应公开详细的使用数据，包括提示缓存字段，这些字段以天真的计数所遗漏的方式改变了经济性。Google在每个Gemini响应上返回使用元数据。这是有用的运营可见性，但不是治理。

Provider dashboards tell you what the service consumed, but they generally do not tell you which business workflow created the cost, which application feature is responsible, which team owns the spend, or whether the interaction produced meaningful business value. Without that attribution, you are effectively staring at infrastructure telemetry while trying to answer business questions.提供商仪表板告诉你服务消费了什么，但它们通常不会告诉你哪个业务工作流产生了成本、哪个应用功能负责、哪个团队拥有支出，或者交互是否产生了有意义的业务价值。没有这种归属，你实际上是在盯着基础设施遥测，同时试图回答业务问题。

That is why application instrumentation matters.这就是应用仪器化重要的原因。

Instrument the Application Like a Grown-Up像成年人一样对应用进行仪器化

If you care about AI economics, the application needs to participate in the observability story. At minimum, I would want telemetry capturing application name, environment, feature name, workflow identifiers, model selection, token usage (broken out by input, output, cache, and reasoning where the provider exposes that level of information), estimated cost, latency, tool invocation counts, response outcomes, and enough user or session correlation to tie activity back to meaningful business workflows.如果你关心AI经济学，应用需要参与可观测性故事。至少，我希望遥测捕获应用名称、环境、功能名称、工作流标识符、模型选择、token使用情况（按输入、输出、缓存和推理细分，如果提供商公开该级别信息的话）、估计成本、延迟、工具调用次数、响应结果，以及足够的用户或会话关联，以将活动与有意义的业务工作流联系起来。

This is where the conversation shifts from "the AI bill feels weird" to "we can actually explain what happened."这就是对话从“AI账单感觉奇怪”转向“我们实际上可以解释发生了什么”的地方。

Capture Real Usage, Not Vibes捕获真实使用情况，而非感觉

A generic token estimator is fine for explaining the concept, but if you are building something real, you want actual provider-returned usage data. Each of the three major providers expose this information, and each one exposes nuance that naive token counting completely misses. Caching changes the math. Reasoning and "thinking" tokens get billed as output. Cached input tokens are usually a subset of your reported input, not a separate line, so you have to subtract before you multiply (a mistake I have watched double a cost estimate).通用token估算器用于解释概念还可以，但如果你在构建真实的东西，你需要实际的提供商返回使用数据。三大提供商都公开这些信息，每个都公开了天真的token计数完全忽略的细微差别。缓存改变了计算方式。推理和“思考”token作为输出计费。缓存的输入token通常是你报告的输入的子集，而不是单独一行，所以你必须先减去再相乘（我见过这个错误使成本估算翻倍）。

Here is the quick version for each provider. The full, repo-ready version with shared pricing tables and a structured telemetry emitter is linked at the bottom of this blog.以下是每个提供商的快速版本。完整的、仓库就绪版本，包含共享定价表和结构化遥测发射器，链接在本文底部。

Claude (Anthropic)Claude (Anthropic)

Anthropic returns input_tokens, output_tokens, cache_creation_input_tokens, and cache_read_input_tokens as separate fields, which is exactly the kind of granularity AI FinOps needs.Anthropic返回input_tokens、output_tokens、cache_creation_input_tokens和cache_read_input_tokens作为单独字段，这正是AI FinOps需要的粒度。

import anthropic

# === pick your model here ===
MODEL = "claude-opus-4-8"   # current Anthropic flagship (Opus 4.8)

MODEL_PRICING = {
    # USD per 1M tokens. Verify against current Anthropic pricing before billing.
    "claude-opus-4-8": {
        "input": 5.00,
        "output": 25.00,
        "cache_write_5m": 6.25,   # 1.25x input
        "cache_read": 0.50,       # 0.1x input
    }
}

def estimate_claude_cost(model: str, usage) -> float:
    p = MODEL_PRICING[model]
    input_tokens = getattr(usage, "input_tokens", 0) or 0
    output_tokens = getattr(usage, "output_tokens", 0) or 0
    cache_write = getattr(usage, "cache_creation_input_tokens", 0) or 0
    cache_read = getattr(usage, "cache_read_input_tokens", 0) or 0
    return round(
        (input_tokens / 1_000_000) * p["input"]
        + (output_tokens / 1_000_000) * p["output"]
        + (cache_write / 1_000_000) * p["cache_write_5m"]
        + (cache_read / 1_000_000) * p["cache_read"],
        6,
    )

client = anthropic.Anthropic()
resp = client.messages.create(
    model=MODEL,
    max_tokens=512,
    messages=[{"role": "user", "content": "Summarize this contract change."}],
)
print(estimate_claude_cost(MODEL, resp.usage))

Note that with Claude, input_tokens already excludes cached reads and cache writes, so you add the four buckets together rather than subtracting. That is the opposite of how OpenAI and Gemini report it, which is exactly the kind of detail that wrecks a spreadsheet built on assumptions.注意，对于Claude，input_tokens已经排除了缓存读取和缓存写入，所以你应该将四个桶相加而不是相减。这与OpenAI和Gemini的报告方式相反，正是那种会摧毁基于假设的电子表格的细节。

ChatGPT (OpenAI)ChatGPT (OpenAI)

OpenAI reports total prompt_tokens (which already includes any cached tokens) plus a cached_tokens breakdown, and reasoning tokens roll into completion_tokens. So cached input is a subset you have to back out before pricing.OpenAI报告总prompt_tokens（已经包含任何缓存token）加上cached_tokens细分，推理token计入completion_tokens。因此缓存输入是你在定价前必须剔除的子集。

from openai import OpenAI

# === pick your model here ===
MODEL = "gpt-5.5"   # current OpenAI flagship (GPT-5.5)

MODEL_PRICING = {
    # USD per 1M tokens. Verify against current OpenAI pricing before billing.
    "gpt-5.5": {
        "input": 5.00,
        "output": 30.00,
        "cached_input": 0.50,   # 0.1x input
    }
}

def estimate_openai_cost(model: str, usage) -> float:
    p = MODEL_PRICING[model]
    prompt = usage.prompt_tokens or 0
    completion = usage.completion_tokens or 0  # reasoning tokens included here
    details = getattr(usage, "prompt_tokens_details", None)
    cached = getattr(details, "cached_tokens", 0) or 0
    uncached_input = prompt - cached
    return round(
        (uncached_input / 1_000_000) * p["input"]
        + (cached / 1_000_000) * p["cached_input"]
        + (completion / 1_000_000) * p["output"],
        6,
    )

client = OpenAI()
resp = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "Summarize this contract change."}],
)
print(estimate_openai_cost(MODEL, resp.usage))

Gemini (Google)Gemini (Google)

Google's current SDK is google-genai (the older google-generativeai package is on its way out). Usage comes back on response.usage_metadata. Google当前的SDK是google-genai（较旧的google-generativeai包正在淘汰）。使用情况通过response.usage_metadata返回。

Two things to watch: cached_content_token_count is a subset of prompt_token_count, and thoughts_token_count (thinking) is billed at the output rate, so it gets added to your output bucket, not your input one. One more for the Pro tier: Gemini 3.1 Pro uses context-tier pricing, where rates step up once input crosses 200K tokens (the quick snippet below shows the standard tier; the repo handles the step-up).两件事要注意：cached_content_token_count是prompt_token_count的子集，thoughts_token_count（思考）按输出费率计费，所以它应加入输出桶，而不是输入桶。对于Pro层级还有一点：Gemini 3.1 Pro使用上下文层级定价，一旦输入超过20万token，费率会阶梯上升（下面的快速片段显示标准层级；仓库处理阶梯上升）。

from google import genai

# === pick your model here ===
MODEL = "gemini-3.1-pro"   # current Gemini flagship (3.1 Pro)

MODEL_PRICING = {
    # USD per 1M tokens, standard context (<=200K). Verify before billing.
    "gemini-3.1-pro": {
        "input": 2.00,
        "output": 12.00,
        "cached_input": 0.20,   # 0.1x input
    }
}

def estimate_gemini_cost(model: str, usage) -> float:
    p = MODEL_PRICING[model]
    prompt = usage.prompt_token_count or 0
    cached = getattr(usage, "cached_content_token_count", 0) or 0
    candidates = usage.candidates_token_count or 0
    thoughts = getattr(usage, "thoughts_token_count", 0) or 0
    uncached_input = prompt - cached
    billable_output = candidates + thoughts  # thinking billed as output
    return round(
        (uncached_input / 1_000_000) * p["input"]
        + (cached / 1_000_000) * p["cached_input"]
        + (billable_output / 1_000_000) * p["output"],
        6,
    )

client = genai.Client()
resp = client.models.generate_content(
    model=MODEL,
    contents="Summarize this contract change.",
)
print(estimate_gemini_cost(MODEL, resp.usage_metadata))

Three providers, three slightly different ways of reporting the same idea, and three different places where a naive estimate could potentially go wrong. The pattern that scales is the same regardless of provider: wrap the SDK call, capture the actual usage metadata, enrich it with application context (feature, workflow, team, environment), and emit a structured telemetry event.三个提供商，三种略有不同的方式报告同一个概念，三个不同的地方可能使天真的估算出错。可扩展的模式无论提供商如何都相同：封装SDK调用，捕获实际使用元数据，用应用上下文（功能、工作流、团队、环境）丰富它，并发出结构化遥测事件。

Azure Teams: KQL Is Still Your FriendAzure团队：KQL仍然是你的朋友

If you are an Azure-heavy shop, this becomes operationally straightforward, because AI telemetry can land in Log Analytics alongside broader application observability. Emit a custom event per call and the queries write themselves.如果你是一个重度Azure环境，这在操作上变得直接，因为AI遥测可以与更广泛的应用可观测性一起落入Log Analytics。每次调用发出自定义事件，查询就会自动编写。

customEvents
| where name == "AIUsageEvent"
| extend Provider = tostring(customDimensions.provider)
| extend Feature = tostring(customDimensions.feature)
| extend Model = tostring(customDimensions.model)
| extend TotalTokens = todouble(customDimensions.total_tokens)
| extend EstimatedCost = todouble(customDimensions.estimated_cost_usd)
| summarize
    Requests = count(),
    TotalTokens = sum(TotalTokens),
    EstimatedCost = sum(EstimatedCost)
    by Provider, Feature, Model, bin(timestamp, 1d)
| order by EstimatedCost desc

Now you can identify expensive workflows instead of staring vaguely at provider billing reports and hoping insight arrives.现在你可以识别昂贵的工作流，而不是模糊地盯着一堆提供商计费报告并希望洞察降临。

Useful Tools Worth Exploring值得探索的有用工具

If you are building this seriously, several platforms are worth a look:如果你在认真构建这个，有几个平台值得一看：

• Langfuse for tracing, prompt analytics, and cost attribution.Langfuse：用于追踪、提示分析和成本归属。

• Helicone for request visibility, caching, and spend tracking - note Helicone has moved into maintenance mode following its aqcquisition by Mintlify, so check whether it still fits before building using this tool.Helicone：用于请求可见性、缓存和支出跟踪——注意Helicone被Mintlify收购后已进入维护模式，所以在使用此工具构建前检查它是否仍然合适。

• OpenLIT for OpenTelemetry-native AI observability. The GitHub repo is located here.OpenLIT：用于OpenTelemetry原生的AI可观测性。GitHub仓库位于此处。

• LiteLLM for proxy-based routing, provider abstraction, and budget controls. Docs are located here. The GitHub repo is located here.LiteLLM：用于基于代理的路由、提供商抽象和预算控制。文档位于此处。GitHub仓库位于此处。

• OpenAI Cookbook for practical implementation examples. The GitHub repo is located here.OpenAI Cookbook：用于实践实现示例。GitHub仓库位于此处。

• FinOps Foundation because AI economics are increasingly part of the broader FinOps conversation.FinOps基金会：因为AI经济日益成为更广泛FinOps对话的一部分。

Final Thought最后思考

AI does not need less experimentation, rather it needs fewer surprise invoices. The organizations that handle this well will not be the ones that panic when spend appears. They will be the ones who architect observability, attribution, and economic accountability into the solution from the beginning. AI不需要更少的实验，而是需要更少的意外账单。处理得好的组织不会是那些在看到支出时恐慌的组织。他们将是从一开始就将可观测性、归属和经济问责制构建到解决方案中的组织。

Because remember...your LLM may have a spending problem. But...more often than not...your architecture does!因为请记住……你的LLM可能有支出问题。但……更多时候……是你的架构有！

Repo with runnable cost-capture code for all three providers, a shared pricing module, and a structured telemetry emitter is available at github.com/sbkuehn/llm-cost-telemetry. Pricing tables verified June 2026; always check the provider pricing pages before you bill anything to anyone.包含适用于所有三个提供商的可运行成本捕获代码、共享定价模块和结构化遥测发射器的仓库位于github.com/sbkuehn/llm-cost-telemetry。定价表已验证至2026年6月；在向任何人收费之前，请始终检查提供商定价页面。