Python Polars 2.1.0: Out of Core Memory Limits and Breaking Arithmetic Change


Python Polars shipped 2.1.0 (tag py-2.1.0) on October 25, 2026. The theme is memory discipline in the out of core engine: file readers now wait for spills while over the memory budget, the default prefetch fraction drops to 0.5, and freed memory returns to the OS during spills. One breaking change lands in eager evaluation: mixed string and numeric arithmetic is no longer allowed (#30071).

The full release notes and downloads are on the GitHub release page. This is not a prerelease. The notes are large, so this post picks the changes that matter if you operate large scans or schedulers around Polars processes.

If you run Polars at the edge of the RAM budget, this release changes what the RSS actually does.

  • File readers wait for spills while over the memory budget (#30296, #30294), so a scan no longer keeps consuming while the budget is already exceeded.
  • The default out of core prefetch fraction is lowered to 0.5 (#30292). Less memory held ahead of time at the cost of more frequent fetches.
  • Freed memory is returned to the OS right away while out of core data is spilled (#30280). Good news for cgroup accounting and for anyone watching container OOM kills.
  • The default spill compression level is set to 1 (#30237), and zstd contexts are reused (#30233), which trades a little CPU for lower spill overhead.
  • The streaming group by now aggregates unique and n_unique in the group by itself (#30281) and runs with shuffles (#29911). The window node can be evaluated out of core (#30282) and supports windows inside filter predicates (#30272).

Concrete takeaway for operators: peak memory and resident memory should move closer together after this release. If you had sizing headroom measured against an older build, remeasure.

  • CSV scans use less memory on wide rows (#30165) and run chunk analysis in parallel (#29818).
  • Cloud CSV and NDJSON scans issue fewer requests with higher concurrency (#29863, #29857), and HTTP object stores are cached (#29757).
  • Nested Parquet level encoding writes directly from level buffers (#30251), string dictionaries are not kept alive for a few selected rows (#30248), and Categorical and Enum dictionary pages filter while decoding (#29882).
  • A quadratic JSON serialization on nested lists and maps is gone (#30203). If your pipelines emit deeply nested NDJSON, this one is worth flagging.

The SQL layer gains real catalog integration.

  • SQLContext can attach Iceberg catalogs via register_catalog (#30113), and the engine uses Iceberg statistics and caches REST catalog scans (#29798, #29790).
  • sink_iceberg adds overwrite_filter and dynamic partition overwrite (#29909) plus create_namespace_if_not_exists (#30087).
  • SQL supports WITH RECURSIVE (#30188, #30166), CURRENT_DATE and friends (#30144), named options on read_parquet and read_ndjson table functions (#29998, #29954), and CREATE [OR REPLACE] TEMP TABLE (#30256).
  • Writing encrypted Parquet files is supported (#29905), and pl.delete_iceberg exists for table removal (#30032).

If you build ELT warehouses on Polars instead of a full database engine, this closes a gap.

Mixed string and numeric arithmetic in eager mode now raises an error (#30071). Pipes that relied on implicit coercion should switch to explicit casts in the expression and verify outputs during the upgrade.