Polars landed 145 commits on main between 25 August and 31 August 2026. The lazy planner now reorders inner joins using scan cardinality, SQL coverage grew for GROUP BY and DATE literals, and leftover 2.0 APIs now fail with a pointer instead of a generic AttributeError. If you collect large star queries or call pl.sql(..., eager=True), check the plan before you ship.
Join reordering now reads scan cardinality ¶
The pola-rs/polars optimizer gained a JOIN_ORDER pass that rewrites runs of inner joins on equality. The planner commit adds the pass, the Python flag, and a test file of 500 lines that writes real parquet so row counts come from scan metadata. The flag is on in the default optimizer mask. The pass is allowed to change which table is built first. Jobs that depended on written table order for memory use can regress until the flag is pinned off.
The tests build a small star schema: a 1000 row fact table, a selective dimension that a filter shrinks, and an unselective dimension that reproduces the fact table. The pass is supposed to join the selective side first. Outer joins and joins that are not equality matches stay in written order. Compare collect with the flag on and off if a query suddenly spills.
lf.collect(optimizations=pl.QueryOptFlags(join_order=False))
A later change feeds scan information into that pass. The old FileInfo row estimate tuple is now typed ScanStats. Cardinality from parquet and IPC metadata fills those stats. When streaming sampling runs out, scan cardinality also picks the join build side. The CSE cache now drops a cached node when recomputing it is cheaper than keeping it.
The Python knob lives in QueryOptFlags. LazyFrame saw the most file churn in this window, because optimizer flags, collect, and 2.0 error paths all sit there.
SQL GROUP BY, DATE literals, and inner product ¶
The SQL crate got two coverage passes and a DATE parse change. context.rs is the hot file.
GROUP BY now keeps compound identifiers distinct when two relations share a column name. ORDER BY can name aggregates that are not in the SELECT list. The planner injects __POLARS_ORDERAGG_* columns and drops them later. Relation lookup is case insensitive.
DATE literals are parsed as temporal values instead of lit(string).cast(Date). Invalid dates fail at parse time. SQL casts got a separate fix. ARRAY_INNER_PRODUCT (alias ARRAY_DOT_PRODUCT) is now a SQL function on equal width Array inputs. List dtype is not converted. Null coordinates do not contribute. If either input array is null for a row, that row is null.
SELECT ARRAY_INNER_PRODUCT(lhs, rhs) AS dot FROM self
An earlier coverage pass added tests for correlated subqueries, EXISTS, window functions, and join wildcard resolution. USING joins and case folding on table names show up there too. That is SQL catching up to the lazy API, not a new engine. A BI tool that quotes identifiers differently than Polars used to fail in ways that looked like missing tables.
2.0 removals fail with a pointer instead of AttributeError ¶
Config.set_auto_structify was already gone. The leftover POLARS_AUTO_STRUCTIFY environment variable is gone too. Setting it to 1 used to pack pl.all() into a struct. The test now asserts two columns remain two columns.
Removed names raise AttributeRemovedError with a replacement. pl.arctan2d points at arctan2 then .degrees(). pl.groups points at a row index plus group_by. pl.read_csv_batched points at scan_csv plus streaming collect. pl.threadpool_size points at thread_pool_size.
The 2.0 rc1 migration guide documents that pl.sql(..., eager=True) now collects through LazyFrame.collect(), so it follows engine affinity (streaming by default) instead of the in memory engine. shift(None) is a hard error. Unknown Arrow extension types load as pl.Extension(name, storage) unless POLARS_UNKNOWN_EXTENSION_TYPE_BEHAVIOR=load_as_storage is set. list.gather, str.contains_any, and str.replace_many no longer implode a flat argument.
Python also gained Expr.name.strip_prefix and strip_suffix. Small, but it matches the existing prefix and suffix helpers. Useful when a join suffix has to come off before a later select.
Sample order, join_where stacks, and parquet pages ¶
Series.sample with replacement and shuffle=False now sorts the sampled indices. Tests assert the output is in original order. If you treated shuffle=False as keep draw order with replacement, results change.
join_where could overflow a rayon worker stack. The cartesian product used unbounded par_iter. Each stolen job adds a stack frame. The fix bounds work to 8 tasks per thread.
Parquet decode of plain pages in a dictionary column failed for categoricals written with use_dictionary=False. That path now round trips.
Duration math in duration_us and duration_ms divides the nanosecond constants first so weeks * NS_WEEK / 1000 does not overflow i64 for long windows.
self.weeks * (NS_WEEK / 1000) + self.nsecs / 1000 + self.days * (NS_DAY / 1000)
Primitive group by aggregations collect into a single chunk instead of one chunk per rayon task. Tests use 20000 rows to catch fragmentation. Iceberg and Delta filter pushdown render PyArrow predicate strings that dataset providers can parse, including isin lists.
What to watch ¶
Join plans can change under the default optimizer. If peak memory moves on a star query, collect once with join_order=False and compare.
Anyone still exporting POLARS_AUTO_STRUCTIFY=1 in job YAML will stop packing structs with no exception. Look for schema width changes.
SQL DATE '...' is stricter, and eager SQL follows streaming collect. Read the 2.0 rc1 upgrade notes before you pin a wheel from main.