Pre-release of Polars 2.0Polars 2.0 预发布版

Today we are releasing the first release candidate for Polars 2.0. The definite 2.0 release will land in the following weeks. We don’t aim to make a big feature release of Polars 2.0. In fact we hope it to be a boring experience for you. The reason we bump this major version is that we can get rid of design decisions made in the past that currently block us and then we want to change defaults to more sensible settings that will benefit a greater audience. The biggest default change will be that all LazyFrame queries now will run on the streaming engine. Casual Polars users can therefore expect huge improvements in memory usage and performance. In aggregate we expect the streaming engine to be easily 5x faster.今天，我们发布了 Polars 2.0 的首个候选版本 (RC)。正式的 2.0 版本将在未来几周内发布。我们并不打算在 Polars 2.0 中推出重大的功能更新，事实上，我们希望这次升级对你来说是一个“平淡无奇”的体验。我们提升主版本号的原因在于，我们需要摒弃过去那些阻碍发展的架构决策，并希望将默认设置调整得更加合理，从而惠及更广泛的用户群体。其中最大的默认设置变更在于：所有的 LazyFrame 查询现在都将默认在流式引擎 (streaming engine) 上运行。因此，普通 Polars 用户将能体验到内存占用和性能方面的巨大提升。总体而言，我们预计流式引擎的速度将轻松提升 5 倍。

To help users transition to 2.0, we have posted a full migration guide. This post will cover a few of the highlights.为了帮助用户顺利过渡到 2.0 版本，我们发布了一份完整的迁移指南。本文将重点介绍其中的一些亮点。

Streaming engine as default默认使用流式引擎

This is the biggest impact change of 2.0. Calling collect on a LazyFrame will now default to the streaming engine, leading to massive memory and performance improvements on most queries for users. The reason this required a major version bump is that the streaming engine doesn’t guarantee row-order by default for certain operations (join, group_by, unpivot, etc.). If you require observable row-order in those operations, you can opt in to that by setting maintain_order=True.这是 2.0 版本中影响最大的变更。现在，对 LazyFrame 调用 collect 时将默认使用流式引擎，这将为大多数用户的查询带来巨大的内存和性能优化。之所以需要提升主版本号，是因为流式引擎在某些操作（如 join、group_by、unpivot 等）中默认不保证行顺序。如果你在这些操作中需要观察行顺序，可以通过设置 maintain_order=True 来开启该功能。

For users who want to keep using the “in-memory” engine as default, they can do so by setting the engine affinity.对于希望继续默认使用“内存内 (in-memory)”引擎的用户，可以通过设置引擎亲和性 (engine affinity) 来实现。

lf = pl.LazyFrame({"k": [2, 1, 0], "v": ["a", "b", "c"]})
other = pl.LazyFrame({"k": [0, 1, 2], "r": ["x", "y", "z"]})

# 2.0: engine="auto" now resolves to the streaming engine.
# Row order is no longer guaranteed for joins, group_by, unpivot, ...
(
    lf
    .join(other, on="k", how="left")
    .collect()
)
# ┌─────┬─────┬─────┐
# │ k   ┆ v   ┆ r   │   <- order may not match `lf`'s original row order
# └─────┴─────┴─────┘

# Opt in to observable order for this query:
(
    lf
    .join(other, on="k", how="left", maintain_order="left")
    .collect()
)

# Or keep the old in-memory engine as the default, process-wide:
pl.Config.set_engine_affinity("in-memory")

# ...or per query:
(
    lf
    .join(other, on="k", how="left")
    .collect(engine="in-memory")
)

Stricter Polars更严格的 Polars

Polars aims to be strict and fail fast. Errors should ideally raise up-front, not 20 minutes into a pipeline. Implicit behavior on data-mismatches should be opt-in, not a default, since those mismatches can hide bugs. This strictness has become even more valuable with the rise of AI-driven development. Agents can validate a query’s structure early by calling collect_schema(), which resolves types and catches schema-level mismatches without materializing any data. This ensures fast feedback for the agents, meaning they can iterate faster. Not all errors can be caught during compilation of the query plan, some depend on data. In these cases Polars defaults to stricter behavior to ensure inconsistencies are caught instead of silently producing different results.Polars 旨在保持严谨并尽早报错。理想情况下，错误应该在第一时间抛出，而不是在流水线运行 20 分钟后才出现。对于数据不匹配的情况，隐式行为应该由用户主动选择，而不是作为默认设置，因为这些不匹配可能会掩盖潜在的 Bug。随着人工智能辅助开发的兴起，这种严谨性变得更加重要。AI 代理可以通过调用 collect_schema() 提前验证查询结构，从而在不实现任何数据的情况下解析类型并捕获模式层面的不匹配。这确保了代理能获得快速反馈，从而加快迭代速度。并非所有错误都能在编译查询计划时被捕获，有些错误取决于具体数据。在这些情况下，Polars 默认采用更严格的行为，以确保不一致性被及时发现，而不是静默地产生错误结果。

Below are a few examples where Polars has gotten more strict:以下是 Polars 变得更加严格的几个示例：

is_in lossless type-coercionis_in 的无损类型强制转换

If you run an is_in expression on different data-types, Polars used to cast both types to their common supertype, even if that conversion was lossy Below is an example with user-ids that can go wrong by silent data-type mismatches.如果你在不同数据类型上运行 is_in 表达式，Polars 过去会将两种类型都转换为它们的公共超类型，即使这种转换是有损的。以下是一个因数据类型静默不匹配而导致用户 ID 出错的示例。

# Checking if a user ID matches a list of "flagged" account IDs
# (flagged_ids loaded from a JSON export, where large IDs became floats)
flagged_ids = pl.Series([9007199254740992.0])
user_id = pl.Series([9007199254740993])  # Int64 -> a different ID, off by 1
user_id.is_in(flagged_ids)

Before 2.0, user_id gets coerced to Float64 to match flagged_ids. But 9007199254740993 sits above 2^53 (9007199254740992), the largest integer float64 can represent exactly, so it silently rounds down to 9007199254740992.0, giving a false positive.在 2.0 之前，user_id 会被强制转换为 Float64 以匹配 flagged_ids。但 9007199254740993 大于 2^53 (9007199254740992)，这是 Float64 能精确表示的最大整数，因此它会静默舍入到 9007199254740992.0，从而导致误报。

In 2.0 this raises: InvalidOperationError: 'is_in' cannot check for Int64 values in List(Float64) data., users should explicitly cast to deal with lossy type conversion.在 2.0 中，这会抛出错误：InvalidOperationError: 'is_in' cannot check for Int64 values in List(Float64) data.，用户应显式进行类型转换以处理有损转换。

Strict concatenation严格的连接 (Concatenation)

Horizontal concat will now check lengths instead of silently filling with null.水平连接 (Horizontal concat) 现在会检查长度，而不是静默地用 null 填充。

# Joining per-day transaction counts with per-day fraud-flag counts,
transactions = pl.DataFrame({"day": [1, 2, 3, 4, 5], "count": [120, 98, 143, 87, 156]})

# Upstream job for day 5 failed silently
fraud_flags = pl.DataFrame({"flagged": [2, 0, 5, 1]})  # only 4 rows

pl.concat([transactions, fraud_flags], how="horizontal")

shape: (5, 2)
┌─────┬───────┬─────────┐
│ day ┆ count ┆ flagged │
│ 1   ┆ 120   ┆ 2       │
│ 2   ┆ 98    ┆ 0       │
│ 3   ┆ 143   ┆ 5       │
│ 4   ┆ 87    ┆ 1       │
│ 5   ┆ 156   ┆ null    │  <- day 5 silently has no flag count
└─────┴───────┴─────────┘

In 2.0 this will raise with:在 2.0 中，这会抛出以下错误：

ShapeError: cannot concat dataframes with different heights in 'strict' mode

If padding is what you wanted, you have to explicitly opt-in to that with how="horizontal_extend". Making that intention clear to the reader.如果你确实需要填充，则必须通过 how="horizontal_extend" 显式选择。这能让你的意图对阅读代码的人更加清晰。

Removal of casts in favor of dedicated methods/constructors移除强制转换，改用专门的方法/构造函数

Another one worth mentioning is the removal of many casts that were ambiguous or should be applied via their dedicated parsing expression, leading to one obvious way to parse data.另一个值得一提的变更是移除了许多模棱两可的强制转换，这些转换现在应通过专门的解析表达式来应用，从而确保解析数据有一种明确的方式。

Enums/Categoricals <> integers.枚举/分类 (Enums/Categoricals) 与整数的转换。

pl.Series([None, 1, 0, 2], dtype=pl.UInt32).cast(pl.Enum(["a", "b", "c"]))
# ComputeError: casting from u32 to enum is not supported.

Use instead: .cat.to(dtype) for int → categorical, .cat.physical() for categorical → int.请改用：.cat.to(dtype) 将整数转为分类，使用 .cat.physical() 将分类转为整数。

Parsing Strings to temporal data-types将字符串解析为时间数据类型

pl.Series(["2022-08-30"]).cast(pl.Date)
# InvalidOperationError: casting from string to date is not supported.

Use instead: .str.to_date() / .str.to_datetime(). These allow you to apply a parsing format, giving you more control over how the data is parsed.请改用：.str.to_date() / .str.to_datetime()。这些方法允许你应用解析格式，从而更好地控制数据的解析方式。

These were just a few examples, but we landed many more strictness improvements. See them all in the migration guide.以上仅是几个例子，我们还进行了许多其他严格性改进。请参阅迁移指南以了解全部内容。

Raising informative errors抛出信息丰富的错误

We put a lot of effort into making sure you as user or your agent can continue if you used old parameters that are not supported anymore. We added two new typed exceptions for this; polars.exceptions.AttributeRemovedError and polars.exceptions.ArgumentRemovedError that handle removed attributes and methods and removed parameters respectively.我们投入了大量精力，确保你或你的 AI 代理在使用不再支持的旧参数时，能够获得明确的指引。为此，我们新增了两个类型化异常：polars.exceptions.AttributeRemovedError 和 polars.exceptions.ArgumentRemovedError，分别用于处理已移除的属性/方法以及已移除的参数。

The error messages should point you to the new API instead. Below we show two examples.错误信息应直接指向新的 API。下面我们展示两个示例。

>>> lf.melt(id_vars="a", value_vars="b")
polars.exceptions.AttributeRemovedError: `melt` was removed in version 2.0;
use `LazyFrame.unpivot` instead, with `index` instead of `id_vars`
and `on` instead of `value_vars`

>>> df.join(df, on="a", join_nulls=True)
polars.exceptions.ArgumentRemovedError: the argument 'join_nulls' for
'DataFrame.join' was deprecated in version 1.24 and has been removed
in 2.0.0. It was renamed to 'nulls_equal' in version 2.0.

Most of the removed functionality has been deprecated for a long time and hopefully should not have affected your pipelines if you have stayed up to date. Reach out to us if you think we should have kept some functionality you relied on.大多数已移除的功能早已被弃用，如果你一直保持版本更新，应该不会影响你的流水线。如果你认为我们移除了一些你所依赖的功能，请联系我们。

Last words结语

Polars 2.0 is about better defaults (most importantly the streaming engine) and a better API. We hope this release is rather boring. We don’t gate new features behind major version bumps as we ship them as soon as their ready.Polars 2.0 的核心在于更好的默认设置（最重要的是流式引擎）以及更完善的 API。我们希望这次发布版本能足够“平淡”。我们不会将新功能锁定在主版本更新中，因为一旦功能就绪，我们就会立即发布。

Don’t be mistaken, Polars 2.x will be much better than 1.x. There is a lot in flight that we haven’t talked publicly enough: proper out-of-core support for the streaming engine, a new IO-plugin design, what we think will be the fastest S3 reader out there, major SQL coverage improvements, a cost-based planner, join reordering, and the removal of mmap, which will make our pipelines fully async end to end.别误会，Polars 2.x 将比 1.x 好得多。我们还有许多正在进行中但尚未公开讨论的特性：流式引擎的完善外存 (out-of-core) 支持、全新的 IO 插件设计、我们认为将是目前最快的 S3 读取器、大幅提升的 SQL 覆盖率、基于成本的规划器、连接重排序，以及移除 mmap，这将使我们的流水线实现完全的端到端异步。

Try the release candidate by installing pip install polars==2.0rc1. Give it a spin and reach out to us here: https://github.com/pola-rs/polars/issues or contact us on discord: https://discord.gg/4UfP5cfBE7.通过 pip install polars==2.0rc1 安装并尝试此候选版本。欢迎体验并在此处联系我们：https://github.com/pola-rs/polars/issues 或通过 Discord 与我们沟通：https://discord.gg/4UfP5cfBE7。