Rapidly scaling online storage to serve over 1 billion ChatGPT users

Every OpenAI product depends on fast, reliable access to data, whether someone is logging in, checking their Codex settings, or starting a new conversation in ChatGPT. Each of those actions may require many separate data lookups before the product can respond. If those requests are slow, the product feels slow. If those requests fail, the product stops working entirely.OpenAI 的每一款产品都依赖快速、可靠的数据访问，无论用户是在登录、查看自己的 Codex 设置，还是在 ChatGPT 中开启新对话。其中每个操作都可能需要多次独立的数据查询，产品才能做出响应。如果这些请求很慢，产品就会显得很慢。如果这些请求失败，产品就会完全停止工作。

Habitat is the online storage platform we built so OpenAI products can quickly and reliably access needed information. Habitat now handles more than 70 million requests every second, supporting products used by over 1 billion people each week, across almost 40 geographic regions. Two years ago, Habitat started as a simple Python client-side library connected to a single database. Today, it’s a complex distributed system that serves more than 500 petabytes of data.Habitat 是我们构建的在线存储平台，让 OpenAI 产品能够快速、可靠地访问所需信息。如今，Habitat 每秒处理超过 7000 万次请求，支撑着每周超过 10 亿人使用的产品，覆盖近 40 个地理区域。两年前，Habitat 还只是一个连接到单一数据库的简单 Python 客户端库。今天，它已经是一个复杂的分布式系统，服务超过 500 PB 的数据。

Building and operating infrastructure at this scale is no easy feat, but also not particularly challenging. What made our situation unique is the unprecedented rate at which we’ve had to scale to support staggering user growth and product demand while simultaneously building out a mature platform. Often, system engineers build for 10x scale, and hope for it to hold for a few years while preparing for the next 10x. In our case, we've grown more than 10x year-over-year for the last three years. As a result, building and operating Habitat has been a series of tactical decisions and sequencing: understanding each component at the lowest level to squeeze as much juice out of our existing stack, while fending off storage and compute capacity crunches to buy time for foundational investments.构建并运营这种规模的基础设施绝非易事，但也算不上特别具有挑战性。真正让我们处境独特的是，我们不得不以前所未有的速度扩展，以支撑惊人的用户增长和产品需求，同时还要建设一个成熟的平台。系统工程师通常会按 10 倍规模来构建，并希望它能撑几年，同时为下一个 10 倍做准备。而在我们的情况里，过去三年我们每年都增长超过 10 倍。因此，构建和运营 Habitat 的过程是一系列战术决策和排序：在最底层理解每个组件，从现有技术栈中榨取尽可能多的能力，同时抵御存储和计算容量紧缺，为基础性投入争取时间。

As OpenAI grew, Habitat had to grow with it: first by becoming reliable enough for mission-critical product traffic, then fast enough for global users, and finally, to deftly operate at massive scale. This post is the first in a two-part series on how we scaled online storage. In this post, we’ll share how Habitat evolved, why we turned it from a library into a service, and how we stretched a service written in an uncommon serving stack language—Python—into a reliable storage platform layer.随着 OpenAI 成长，Habitat 也必须随之成长：首先要变得足够可靠，以承载关键任务的产品流量；然后要足够快，以服务全球用户；最后还要能够灵巧地在大规模下运行。本文是两部分系列文章的第一篇，讲述我们如何扩展在线存储。在这篇文章中，我们将分享 Habitat 如何演进、我们为什么把它从一个库变成一个服务，以及我们如何把一个用不常见的服务栈语言——Python——编写的服务，扩展成可靠的存储平台层。

In a future post, we’ll go into detail about how we made multi-tenancy reliability at scale, our layered strategy for optimizing read performance, and how we scaled our partnership with Azure Cosmos DB to reliably handle unprecedented demand.在后续文章中，我们将详细讲述我们如何在大规模下实现多租户可靠性、我们优化读取性能的分层策略，以及我们如何扩展与 Azure Cosmos DB 的合作，以可靠地应对前所未有的需求。

Habitat started from a simple idea: product engineers shouldn’t need to think about database management. Habitat began in mid-2024 as a small Python library that interacted with ChatGPT’s main server. It supported a small set of operations that mapped under the hood to the database application, Azure Cosmos DB.Habitat 始于一个简单的想法：产品工程师不应该需要操心数据库管理。Habitat 在 2024 年年中起步时，只是一个与 ChatGPT 主服务器交互的小型 Python 库。它支持一小部分操作，这些操作在底层映射到数据库应用 Azure Cosmos DB。

The library’s job was to give product teams a simple way to store and retrieve data without needing to master the underlying details. Habitat took care of the necessary work: figuring out what kind of data was involved, where it should come from (or go), whether the request was allowed, and so on.这个库的职责是为产品团队提供一种简单的方式来存储和检索数据，而无需掌握底层细节。Habitat 会处理必要的工作：判断涉及哪种数据、数据应该来自（或去往）哪里、请求是否被允许，等等。

Product engineers need not concern themselves with schema lookup, routing, authorization, encryption, serialization, request shaping, and connection pooling. They didn’t even need to consider where the data comes from: Azure Cosmos DB, caches, or other types of storage.产品工程师无需关心模式查找、路由、授权、加密、序列化、请求整形和连接池。他们甚至不需要考虑数据来自哪里：Azure Cosmos DB、缓存，还是其他类型的存储。

This Python library worked well and Habitat saw rapid adoption among product engineers at OpenAI, despite no concerted central push away from using self-serve Postgres and Azure Cosmos DB.这个 Python 库运行良好，Habitat 在 OpenAI 的产品工程师中迅速得到采用，尽管当时并没有集中的中央推动来让团队不再使用自助式 Postgres 和 Azure Cosmos DB。

As product needs evolved, it was even easy for product developers to add to the shared library support for features like client-side caching, compression, or encryption.随着产品需求演进，产品开发者甚至可以轻松地为这个共享库添加对客户端缓存、压缩或加密等功能的支持。

By the middle of 2025, Habitat had reached its limits as a client-side implementation. As the Habitat layer had grown more complex and OpenAI’s services count increased, backward-compatible protocol changes had become infeasible.到 2025 年年中，Habitat 作为客户端侧实现已经达到了极限。随着 Habitat 层变得越来越复杂，并且 OpenAI 的服务数量不断增加，保持向后兼容的协议变更已经变得不可行。

In one instance, we wanted to reduce the blast radius of any single region outage for our most critical data sets by migrating them to a set of regionally distributed Azure Cosmos DB accounts. Making this change required introducing extra routing logic into the client, disabled behind a feature flag, ensuring it rolled out to all clients, and then enabling the feature flag.例如，我们曾希望通过将最关键的数据集迁移到一组按区域分布的 Azure Cosmos DB 账户，来缩小任何一个区域故障的影响范围。做出这一变更需要在客户端中引入额外的路由逻辑，先通过功能标志将其禁用，确保它推广到所有客户端，然后再启用该功能标志。

Coordinating deployments across dozens of services and working with each team to roll it out took days. Before enabling this, we realized we wanted to introduce some shadowing to ensure the sharding logic would be correct. That took another couple of days to roll out. A bug fix for something we realized was incorrect? Another couple of days. Eventually, we were ready to enable the flag, only for one of the teams to roll back their service for unrelated reasons to a previously buggy client, causing the outage we had worked so hard to avoid.在几十个服务之间协调部署，并与每个团队合作推广，花了数天时间。在启用之前，我们意识到需要引入一些影子流量来确保分片逻辑正确。这又花了几天时间推广。我们意识到某处不正确，修复一个 bug？又花了几天。最终，我们准备好启用该标志，结果其中一个团队因为无关原因把他们的服务回滚到了一个此前有 bug 的客户端，导致我们一直努力避免的故障发生了。

Changes to the client library necessitated complex coordination across dozens of services, a process that proved increasingly brittle, inefficient, and susceptible to operational failures. To reduce this operational fan out for our future deployments, we decided to pull Habitat into its own service.对客户端库的变更需要在几十个服务之间进行复杂协调，而这个过程被证明越来越脆弱、低效，并且容易发生运维故障。为了减少未来部署中的这种运维扩散，我们决定把 Habitat 抽离成一个独立服务。

By decoupling the storage logic into a standalone service, we established a single point of control for deployments, observability, and platform enhancements. Instead of managing fragmented updates, we could implement improvements centrally, providing immediate benefits to every OpenAI product.通过将存储逻辑解耦为一个独立服务，我们为部署、可观测性和平台增强建立了一个单一控制点。我们不再需要管理碎片化的更新，而是可以集中实施改进，让每个 OpenAI 产品立即受益。

A centralized service also gives us a single chokepoint to provide the strongest data security and privacy primitives. Habitat service is where we can centrally enforce access control policies, perform audit logging, and limit access to underlying storage resources like Azure Cosmos DB. Habitat plays a critical role in protecting user data and preventing unauthorized access from external, internal, and agent actors.集中式服务还为我们提供了一个单一关口，以提供最强的数据安全和隐私原语。Habitat 服务是我们集中执行访问控制策略、进行审计日志记录，并限制对 Azure Cosmos DB 等底层存储资源访问的地方。Habitat 在保护用户数据、防止来自外部、内部和代理行为者的未授权访问方面发挥着关键作用。

We knew we needed a service, but we didn’t want to migrate off Python quite yet, even with Python’s additional overhead as a service. Using Python for a high-throughput service increased network latency and added substantial CPU and memory scaling costs compared to local library execution. Moreover, we recognized that the inefficiencies of Python would not be acceptable at 100x scale, making an eventual rewrite almost certain.我们知道需要一个服务，但还不想完全迁离 Python，即使 Python 作为服务会带来额外开销。与本地库执行相比，使用 Python 运行高吞吐量服务会增加网络延迟，并带来可观的 CPU 和内存扩展成本。此外，我们认识到 Python 的低效在 100 倍规模下将不可接受，因此最终重写几乎不可避免。

However, we viewed this as a strategic incursion of technical debt. Our primary objective then was not cost or resource optimization, but rather unblocking product developers and achieving platform stability. By accepting the performance trade-offs of a Python service in the short term, we were able to prioritize more immediate challenges, establish our core APIs, and build out a robust infrastructure.不过，我们把这视为一次战略性的技术债引入。当时我们的首要目标不是成本或资源优化，而是为产品开发者解除阻塞并实现平台稳定性。通过在短期内接受 Python 服务的性能取舍，我们得以优先处理更紧迫的挑战、建立核心 API，并搭建起稳健的基础设施。

We also made a calculated wager that the rapid advancement of our own coding models would simplify the technical path in the future. We bet that by the time a full migration off Python was required, Codex and GPT would make that migration achievable. That bet eventually proved correct.我们还做了一个经过权衡的押注：我们自己编码模型的快速进步会在未来简化技术路径。我们赌的是，到必须完全迁离 Python 的时候，Codex 和 GPT 会让这次迁移变得可行。这个赌注最终被证明是正确的。

Running Habitat as a Python service would be suboptimal, performance-wise, but a necessary choice. Python lets us move quickly, but it doesn’t mean we could throw caution to the wind and accept meaningfully worse latencies. When the average user request results in hundreds of database calls, the slowest database call is the one the user feels. We’ve found the main challenge in running a Python service at this scale is in managing these tail latencies.从性能角度看，将 Habitat 作为 Python 服务运行并非最优，但却是必要的选择。Python 让我们能够快速行动，但这并不意味着我们可以不顾一切，接受明显更差的延迟。当普通用户请求会导致数百次数据库调用时，用户感受到的是最慢的那次数据库调用。我们发现，在这种规模下运行 Python 服务的主要挑战在于管理这些尾部延迟。

Asyncio helps Python execute I/O-bound workloads concurrently, but does not help work around the Python GIL and provide CPU parallelism. In addition to I/O-heavy request proxying, Habitat handles many CPU-heavy responsibilities and background tasks: routing, compression, encryption, checksumming, downstream health checking, request shadowing, and hedging.Asyncio 帮助 Python 并发执行 I/O 密集型工作负载，但无助于绕开 Python GIL 并提供 CPU 并行性。除了 I/O 密集型的请求代理之外，Habitat 还承担许多 CPU 密集型职责和后台任务：路由、压缩、加密、校验和、下游健康检查、请求影子流量和对冲。

With so many CPU-heavy workloads and background tasks in our service, asyncio scheduling delay can easily dominate tail request latency. Before tuning for our initial service launch, we saw in traces for requests with p99 and higher latency that while downstream storage responded quickly, requests frequently stalled while waiting for the responsible coroutine to be rescheduled to parse the response.我们的服务中有如此多 CPU 密集型工作负载和后台任务，asyncio 调度延迟很容易主导尾部请求延迟。在为初次服务发布进行调优之前，我们在 p99 及更高延迟请求的跟踪中看到，虽然下游存储响应很快，但请求经常停滞，等待负责的协程被重新调度来解析响应。

For Python services at OpenAI, we find that in addition to measuring standard utilization and saturation metrics on memory, CPU, network, and disk usage, it is critical to also monitor the asyncio loop and how busy it is, then tune accordingly.对于 OpenAI 的 Python 服务，我们发现，除了测量内存、CPU、网络和磁盘使用率的标准利用率和饱和度指标之外，监控 asyncio 事件循环及其繁忙程度并据此调优也至关重要。

By periodically scheduling background tasks and recording the delta between expected and actual execution time, we are able to empirically measure event loop scheduling delay in real time. At high utilization, with many expensive tasks, even modest numbers of concurrent requests per process are enough to produce significant scheduling jitter, up to hundreds of milliseconds and in some edge cases several seconds.通过定期调度后台任务，并记录预期执行时间与实际执行时间之间的差值，我们能够实时地以经验方式测量事件循环调度延迟。在高利用率下，面对许多高开销任务，即使每个进程只有少量并发请求，也足以产生显著的调度抖动，最高可达数百毫秒，在某些边缘情况下甚至可达数秒。

As a result, we resort to keeping each process serving only a small number of concurrent requests and instead massively scale out the number of Python worker processes.因此，我们只能让每个进程仅服务少量并发请求，转而大规模横向扩展 Python 工作进程的数量。

In our initial service launch, we discovered through live service CPU profiling one root cause of high asyncio delay (and resulting high tail latencies): periodic JSON parsing of our feature flag configurations via Statsig (a tool that manages feature flags, and can be used to run A/B tests and more).在初次服务发布中，我们通过实时服务 CPU 性能剖析发现了高 asyncio 延迟（以及由此导致的高尾部延迟）的一个根本原因：通过 Statsig（一个管理功能标志、可用于运行 A/B 测试等工作的工具）定期对功能标志配置进行 JSON 解析。

By default, Statsig was configured to poll for refreshed configs every minute with no jitter, and the config included every production rule across every service. Elsewhere, an architectural decision was made to run up to 8 Python processes per pod to push higher CPU usage and provide lower latencies. Combined, this meant that every minute each pod would have some moment where all of its workers stalled processing in-flight requests and instead would spend their CPU cycles parsing a giant configuration file.默认情况下，Statsig 被配置为每分钟轮询一次刷新后的配置，且没有抖动，而配置中包含了每个服务的每条生产规则。另一方面，一项架构决策要求每个 pod 最多运行 8 个 Python 进程，以提高 CPU 使用率并提供更低延迟。两者结合意味着，每分钟每个 pod 都会有某个时刻，其所有工作进程都停止处理进行中的请求，转而把 CPU 周期花在解析一个巨大的配置文件上。

The fix was straightforward once CPU profiling helped us root cause the issue: deploy a smaller targeted config, lengthen the refresh interval, and add some jitter to background tasks like these.一旦 CPU 性能剖析帮助我们定位了根本原因，修复就很直接：部署更小、更有针对性的配置，延长刷新间隔，并为这类后台任务添加一些抖动。

In order to maintain low asyncio delay, it is also critical to maintain good load balancing of requests across server processes; connection pooling can end up being antithetical to this without tuning as well.为了保持低 asyncio 延迟，维持服务器进程之间良好的请求负载均衡也很关键；如果不加调优，连接池最终可能会与这一目标背道而驰。

With client-side connection pooling, a single client process that does many concurrent requests might establish only a handful of server connections and as a result send all of its load to only a handful of processes. Prior to adjusting how we do load balancing, our service had a wide variance of utilization with some tail processes serving 5-10x the number of concurrent requests as the average.使用客户端连接池时，一个发出许多并发请求的客户端进程可能只建立少量服务器连接，结果把所有负载都发送到少数几个进程。在调整负载均衡方式之前，我们的服务利用率差异很大，一些尾部进程服务的并发请求数量是平均值的 5 到 10 倍。

We discovered this in a chance incident where, despite stopping the client that was overloading part of our service, a subset of processes remained degraded well past the bursty traffic. In fact, we noticed those processes experienced runaway degradation, receiving increasingly more requests until we restarted them. Once a pod became overloaded, some behavior was pinning more traffic onto the overloaded pod. This was a class of failures some of our teammates were well-acquainted with from prior work: metastable failure⁠(opens in a new window).我们是在一次偶然事件中发现这一点的：尽管我们已经停止了那个让部分服务过载的客户端，但一部分进程在突发流量过去很久之后仍处于降级状态。事实上，我们注意到这些进程出现了失控式降级，收到的请求越来越多，直到我们重启它们。一旦某个 pod 过载，某些行为就会把更多流量钉到过载的 pod 上。这是一类我们一些队友在以往工作中很熟悉的故障：亚稳态故障（在新窗口中打开）。

We suspected the connection pool was to blame and tested this suspicion by capping max connection reuse duration, which indeed limited the degradation and confirmed our investigation direction. Further investigation found that Python’s aiohttp TCPConnector defaults to LIFO connection reuse: the most recently returned connection is selected for the next request. This is normally a reasonable default: reusing recent connections allows the extra connections created to handle bursty traffic to idle timeout, reducing overhead to maintaining extra connections. In this case, it created a metastable failure for us. During a burst of requests, requests to slower overloaded servers returned connections to the pool later and were therefore selected more frequently by subsequent requests, gradually concentrating more traffic on the pods already struggling. Patching the connection pool to use FIFO reuse broke this feedback loop and even reduced our steady state request variance as well.我们怀疑连接池是罪魁祸首，并通过限制最大连接复用时长来验证这一怀疑，这确实限制了降级，也确认了我们的调查方向。进一步调查发现，Python 的 aiohttp TCPConnector 默认使用 LIFO 连接复用：最近返回的连接会被选中用于下一个请求。通常这是一个合理的默认行为：复用较新的连接，可以让为处理突发流量而创建的额外连接空闲超时，从而降低维护额外连接的开销。但在我们的情况中，它造成了亚稳态故障。在一波请求突发期间，发往较慢的过载服务器的请求会更晚把连接返还到池中，因此会被后续请求更频繁地选中，逐渐把更多流量集中到那些本已苦苦支撑的 pod 上。把连接池修补为使用 FIFO 复用打破了这一反馈循环，甚至还降低了我们稳态下的请求方差。

Today, we mostly depend on Istio and Envoy to provide connection pooling and better server-load-aware balancing strategies throughout OpenAI infrastructure and avoid this problem altogether.如今，我们主要依赖 Istio 和 Envoy 在 OpenAI 基础设施中提供连接池和更好的服务器负载感知均衡策略，从而完全避免这个问题。

One side effect of tuning for low asyncio delay and having so many Python processes is that it becomes very easy to overwhelm downstream dependencies with the vast number of connections (known as a “thundering herd”).为降低 asyncio 延迟而调优，并且拥有如此多 Python 进程的一个副作用是：庞大的连接数量很容易压垮下游依赖（即所谓的“惊群”）。

A regular daily deployment—if not tuned to be slow—can cause significant CPU churn from connection cycling. Or a connection leak can take out the network by saturating the NAT gateway. These are not uncommon problems for other services too, but the threshold for triggering is lowered significantly by having an order of magnitude more processes, often saturating network related resources that clients are not expecting to need to handle in a steady state based on pure throughput alone.一次常规的每日部署——如果没有调优以放慢速度——就可能因为连接不断循环而造成显著的 CPU 波动。或者，连接泄漏可能因打满 NAT 网关而拖垮网络。对其他服务来说，这些问题也并不罕见，但由于进程数量多了一个数量级，触发阈值显著降低，常常会打满网络相关资源，而客户端仅基于纯吞吐量时，并不预期在稳态下需要处理这些资源。

We also rely on Envoy to maximize our connection fan-in. We use it to upgrade Python’s HTTP/1 connections to HTTP/2 to take advantage of multiplexing and then to pool those connections and extend connection lifetimes. Envoy also gives us a central place to implement rate limits and circuit breakers that would be less effective in each standalone Python process.我们还依靠 Envoy 来最大化连接扇入。我们用它把 Python 的 HTTP/1 连接升级为 HTTP/2，以利用多路复用，然后对这些连接进行池化并延长连接生命周期。Envoy 还为我们提供了一个集中位置来实现限流和熔断，而这些如果放在每个独立的 Python 进程中效果会差很多。

One reason we could scale Python this far was Habitat’s constrained API, which keeps request cost predictable. Rather than allowing clients to construct arbitrary SQL queries that could result in large table scans or joins across many tables, Habitat exposes a simple NoSQL API. The lack of a powerful API is an explicit tradeoff in Habitat’s design.我们能够把 Python 扩展到如此程度的一个原因是 Habitat 受限的 API，它让请求成本保持可预测。Habitat 不允客户端构造任意 SQL 查询——那可能导致大规模表扫描或跨多表连接——而是暴露一个简单的 NoSQL API。缺少强大的 API 是 Habitat 设计中的一个明确取舍。

We aim to optimize for simple, predictable, constant-work requests. In our experience, these systems are substantially easier to scale and difficult to get wrong or misuse. Requests with unpredictable fanout are operationally dangerous: they complicate isolation, load balancing, and introduce latency cliffs that are hard to scale for both the service and its clients.我们的目标是针对简单、可预测、工作量恒定的请求进行优化。根据我们的经验，这类系统要容易扩展得多，也很难出错或被误用。扇出不可预测的请求在运维上很危险：它们会使隔离、负载均衡复杂化，并引入延迟悬崖，无论对服务还是其客户端来说都难以扩展。

Before we moved to Habitat and Azure Cosmos DB, most of OpenAI’s online data was stored on Postgres. At that time it was easy to review all query and schema changes to make sure they were well-behaved and operated against indexed data before shipping to production. As the team and products grew, this quickly became unmanageable and was a frequent cause of outages where a single expensive new query on a hot path took out the database.在我们迁移到 Habitat 和 Azure Cosmos DB 之前，OpenAI 的大部分在线数据都存储在 Postgres 上。当时，审查所有查询和模式变更，确保它们行为良好并在上线生产前针对已索引数据运行，是很容易的。随着团队和产品增长，这很快变得无法管理，并频繁导致故障：热路径上一个昂贵的新查询就能拖垮数据库。

The problem here is in cost imbalance: it is cheap and easy to write SQL queries that are expensive and hard to run. In Habitat, we avoid this and make expensive queries exceedingly obvious client-side. There are no unbounded queries that can overload Habitat and complex joins and graph traversals require product teams to do some of the heavy-lifting which helps overall optimize for more efficient designs.这里的问题在于成本不匹配：编写昂贵且难以运行的 SQL 查询既便宜又容易。在 Habitat 中，我们避免了这一点，并让昂贵查询在客户端侧变得极其明显。不存在可能让 Habitat 过载的无界查询，而复杂连接和图遍历要求产品团队承担一部分繁重工作，这有助于整体上优化出更高效的设计。

Habitat exposes a NoSQL API modeled around client-defined object and edge types, inspired by TAO⁠(opens in a new window). Clients predefine objects and edges and how they relate to each other, but not the content of each type. The resulting relationships resemble a graph, but Habitat itself does not support typical graph traversal queries outside of querying direct edges of a particular object.Habitat 暴露一个 NoSQL API，其模型围绕客户端定义的对象和边类型，灵感来自 TAO（在新窗口中打开）。客户端预定义对象和边以及它们彼此之间的关系，但不定义每种类型的内容。由此产生的关系类似图，但 Habitat 本身除了查询某个对象的直接边之外，并不支持典型的图遍历查询。

We partition this graph so that each object and its corresponding edges are colocated in a storage-level partition, but we make no concerted database-level effort to colocate objects and the remote objects to which their edges point. The result is that the model easily partitions for horizontal scalability, but graph traversals are inefficient since any particular hop between objects may require fetching from two entirely different Azure Cosmos DB accounts stored in different regions.我们对这个图进行分区，使每个对象及其对应的边共置于同一个存储级分区中，但我们并没有在数据库层面刻意将对象与其边指向的远程对象共置。结果是，该模型易于分区以实现横向扩展，但图遍历效率低下，因为对象之间的任何一次特定跳转都可能需要从存储在不同区域的两个完全不同的 Azure Cosmos DB 账户中获取数据。

For clients with more complex querying needs, we do provide an offline secondary view of Habitat exposed via Rockset. We use change data capture (CDC) to stream changes from the online storage out to isolated Rockset instances in near-real-time. Each client team is responsible for scaling their own Rockset instance for their complex querying needs.对于查询需求更复杂的客户端，我们确实通过 Rockset 提供了一个 Habitat 的离线二级视图。我们使用变更数据捕获（CDC）将近乎实时的变更从在线存储流式传输到隔离的 Rockset 实例。每个客户端团队负责为自己的复杂查询需求扩展各自的 Rockset 实例。

This Rockset provisioning introduces extra friction to our clients, but we think is the right tradeoff to make at this particular moment: making simple queries the default while providing an escape hatch for those who need complex queries. This design isolates our online storage from read-heavy analytical and search workloads.这种 Rockset 配置给我们的客户端带来了额外摩擦，但我们认为在当下这是正确的取舍：让简单查询成为默认方式，同时为需要复杂查询的人提供一个逃生通道。这种设计将我们的在线存储与读取繁重的分析和搜索工作负载隔离开来。

Deferring a Python rewrite for a year allowed us to focus on more urgent and impactful challenges during our hypergrowth. With the platform maturing and our growth continuing to accelerate, and being the second largest service by core count at OpenAI (and fourth for our Envoy footprint), it was finally time to move past Python. At its peak, Python helped us serve more than 20 million requests every second.将 Python 重写推迟一年，让我们能够在超高速增长期间专注于更紧迫、更有影响力的挑战。随着平台逐渐成熟、增长持续加速，并且按核心数计算它已是 OpenAI 第二大服务（按 Envoy 占用则是第四大），终于到了超越 Python 的时候。在巅峰时期，Python 帮助我们每秒服务超过 2000 万次请求。

In Q2 2026, with just 2 engineers, Codex, and GPT‑5.5, we were able to rewrite the entire service in Rust. This new Rust service is now handling 95% of our production requests; we’ll be deprecating Python entirely in the coming weeks. Our data shows the Rust service is 6x more CPU efficient and 15x more memory efficient than the Python version, with significantly lower average and tail latencies. We plan to share more learnings in a future blog.在 2026 年第二季度，仅凭 2 名工程师、Codex 和 GPT‑5.5，我们就用 Rust 重写了整个服务。这个新的 Rust 服务目前处理我们 95% 的生产请求；未来几周我们将完全弃用 Python。我们的数据显示，Rust 服务比 Python 版本 CPU 效率高 6 倍、内存效率高 15 倍，并且平均延迟和尾部延迟都显著更低。我们计划在未来的博客中分享更多经验。

The Python—and now Rust—service is only one facet of Habitat. In part II of this series explaining how we rapidly scaled our online storage to serve over 1 billion ChatGPT users, we’ll talk about the storage layer and how Habitat serves more than 500 petabytes and over 70 million requests every second.Python——以及现在的 Rust——服务只是 Habitat 的一个方面。在本系列第二部分中，我们将解释我们如何快速扩展在线存储来服务超过 10 亿 ChatGPT 用户，并讨论存储层，以及 Habitat 如何服务超过 500 PB 数据并每秒处理超过 7000 万次请求。

If you want to work on OLTP systems at frontier scale and are interested in this kind of engineering, check out this open role on our team⁠.如果你想在前沿规模上从事 OLTP 系统工作，并对这类工程感兴趣，请查看我们团队的这个开放职位。