Polars 2.0 Upgrade Guide [2026]: Streaming Default + CI Bench

Polars 2.0 Upgrade Guide: Breaking Changes, Streaming Default, and a CI Regression Harness

One pip install -U polars later, your tests start failing. Not because your logic is wrong. Because your rows came back in a different order.

If you already searched for “polars 2.0 upgrade guide”, that’s probably why you’re here.

You’ll leave this guide with two things working:

1) your codebase running on Polars 2.0 (or the 2.0 RC) without “mystery diffs”, and 2) a repeatable regression harness that measures wall time, CPU time, and peak memory (RSS) for your real queries.

Polars 2.0 is a rare “good breaking change”. It makes the fast path the default. The price is that you now have to be explicit about correctness requirements you were accidentally getting for free.

What is Polars 2.0 (and why does the upgrade feel scary)?

Polars 2.0 is a major version of the Polars DataFrame library that changes defaults and tightens type behavior, most notably by running LazyFrame queries on the streaming engine by default.

The anxiety comes from one specific sentence in the official announcement. As Ritchie Vink (creator/maintainer of Polars) explains, in Polars 2.0 calling `collect()` on a `LazyFrame` now defaults to the streaming engine, and that engine does not guarantee row order for some operations unless you opt in.

That’s the trade.

• You get real wins in memory and throughput. The announcement claims the streaming engine can be “easily 5x faster in aggregate” for many lazy queries.
• You lose the comforting illusion that output order will “probably” match input order for `join`, `group_by`, `unpivot`, and friends.

If you have order-dependent tests, dashboard diffs, or downstream code that quietly assumes “left table order survives a join”, you will feel this upgrade.

Streaming engine as default: the Lazy API now streams unless you stop it

The headline breaking change from the upgrade docs is literal: “The Lazy API defaults to the streaming engine” (Polars 2.0-rc upgrade guide).

Concretely:

• Polars 1.x: lf.collect() typically used the in-memory engine unless you opted into streaming.
• Polars 2.0: lf.collect() with engine="auto" resolves to streaming by default.

How do I keep old behavior (in-memory engine) for `collect()`?

You have three levers. Use the smallest hammer that fits.

1) Per call (most explicit):

out = lf.collect(engine="in-memory")

2) Per query chain (useful when you have a couple of known order-sensitive pipelines):

out = (
    lf
    .join(other, on="k", how="left")
    .collect(engine="in-memory")
)

3) Process-wide engine affinity (fine for a staged rollout or a notebook session):

import polars as pl
pl.Config.set_engine_affinity("in-memory")

That last one is straight out of the announcement example from Ritchie Vink.

My opinion: don’t flip global affinity in production just to keep tests passing. That’s a safety blanket, not a fix. The whole point of Polars 2.0 is that streaming is the default for a reason.

When does streaming change row order, and how do I maintain order?

Row order changes when the streaming engine parallelizes or re-chunks data in a way that’s valid for the result but not stable relative to the input.

As Ritchie Vink calls out, row order is not guaranteed for certain operations like:

• join
• group_by
• unpivot

Why did my join/group_by output order change after upgrading?

Because you were relying on an implementation detail.

In Polars 1.x, a lot of pipelines looked stable because the in-memory engine often preserved a left-to-right vibe. With streaming as the default, the engine is allowed to reorder during joins and aggregations to hit performance and memory goals.

If you need stable, observable order, Polars 2.0 makes you say so.

Maintain order only where it matters

For joins, opt in with maintain_order:

out = (
    lf
    .join(other, on="k", how="left", maintain_order="left")
    .collect()
)

That mirrors the official example in the 2.0 pre-release post.

For aggregations, you have a few options depending on what you mean by “order”:

• If you want deterministic output ordering for group keys, explicitly sort:

out = (
    lf
    .group_by("country")
    .agg(pl.col("revenue").sum().alias("rev"))
    .sort("country")
    .collect()
)

• If you want “preserve input order semantics”, stop using a group_by output order as a proxy for that. Decide what the ordering rule actually is (timestamp, key priority, revenue descending, whatever) and encode it.

A practical rule I use in code review: if a test asserts row order after a `join`/`group_by`, it must also assert an explicit ordering rule. Otherwise you’re testing vibes.

Breaking changes checklist: what actually breaks in real codebases

The upgrade docs are exhaustive. Your team doesn’t need exhaustive. Your team needs a short list that prevents the three dumbest failure modes.

Here’s the path I’d take if I were upgrading a production pipeline.

1. Pin versions and test the RC first. GitHub releases already list Python Polars 2.0.0-rc.1 and the last 1.44.x line (Polars releases). Don’t “pip install -U polars” blind.
2. Audit order-dependent tests (joins, group_bys, unpivot). Fix them with maintain_order or explicit .sort().
3. Stop assuming `pl.read_csv` is purely eager (it now dispatches to scan + collect).
4. Replace `LazyFrame.profile()` usage.
5. Run your regression harness on representative datasets. Block merges if you cross a threshold.

That’s the work. Everything else is cleanup.

CSV reading changes and eager/lazy surprises (read_* now dispatches to scan_*().collect())

IO is where upgrades love to hide landmines.

Polars 2.0 changes eager reads:

• pl.read_csv is now dispatched to pl.scan_csv(...).collect() (upgrade guide).
• pl.read_ipc is now dispatched to pl.scan_ipc(...).collect() for all non-use_pyarrow inputs (same doc).

What should I change if my code uses `pl.read_csv` / `pl.read_ipc`?

If your code was relying on:

• file-like object behavior
• “eager read happens now” side effects
• subtle schema inference timing

…make the intent explicit.

If you actually want eager semantics

You can keep read_csv, but mentally treat it as “implemented via lazy scan + collect now”. If you need to control streaming vs in-memory behavior, do it explicitly via the lazy path:

df = pl.scan_csv(path).collect(engine="in-memory")

If you want the benefits of lazy planning

Lean into scanning and do the obvious pushdowns before you collect:

lf = pl.scan_csv(path)
# add filters/projections before collect
out = (
    lf
    .filter(pl.col("status") == "active")
    .select(["user_id", "country"])
    .collect()
)

Also note a smaller CSV change from the docs: infer_schema_files default is now 10. If your dataset has weird “rare” types that only show up after file 10, you can absolutely see schema shifts unless you pin schema or bump that number.

Stricter Polars: type coercion, concat/union, and removed casts

Polars has always been strict. Polars 2.0 is stricter, and I’m fine with that.

The official announcement frames it as “fail fast” rather than letting pipelines run for 20 minutes before raising (same pre-release post).

From the upgrade guide, the big “stricter” buckets you’ll actually notice:

`is_in()` strict coercion (lossy casts no longer happen)

Polars 2.0 makes coercion casts for is_in() strict instead of lossy. If you were comparing values across types and relying on silent conversion, expect failures.

Fix: normalize types up front. This is boring work. It’s also the correct work.

`pl.concat()` / `pl.union()` strict behavior

The upgrade guide calls out “Update the strict behavior of pl.concat()/pl.union().” If you were concatenating frames with mismatched schemas and expecting a “best effort”, Polars will now force you to be explicit.

Fix: align schemas (add missing columns, cast dtypes) before concat.

Casts and operations no longer supported

This is the “you were doing a thing that looked convenient but wasn’t safe” section.

Examples called out in the upgrade guide:

• disabled casting from integers to categoricals (and back)
• removed casts from string to temporal types
• disallowed casting from non-nested types to pl.List(..)
• disallowed boolean operators between booleans and integer types

Yes, it’s painful. But it’s also removing the kind of footguns that create silent data corruption and then eat your weekend.

Raising informative errors: why this helps upgrades (even when it annoys you)

Polars 2.0 treats “Raising informative errors” like a feature, not an afterthought (it’s a full section in the announcement).

In practice, that means:

• errors earlier in planning rather than during execution
• clearer type mismatch messaging
• fewer “it returned something, but not what you thought” situations

I’m pro this change.

I’ve built internal developer tooling (including the SOC 2 scaffolding CLI at Rise People that was adopted org-wide). “Force the right thing early” beats “debug the wrong thing later” every time. The upgrade pain is front-loaded, but the operational cost drops.

Build a repeatable performance regression benchmark harness (wall, CPU, memory)

Here’s the part most upgrade guides conveniently skip. You don’t just want “it seems faster on my laptop.” You want a harness you can run before and after the upgrade, with numbers you can gate in CI.

You want to measure three things:

• Wall time: “how long did it take end-to-end?”
• CPU time: “did we burn more CPU to get that wall time?”
• Peak RSS: “did memory spike and put us into OOM territory?”

Harness design: compare versions, not vibes

A decent harness:

• pins exact versions (ex: polars==1.44.1 vs polars==2.0.0-rc.1)
• runs the same query set on the same dataset
• warms up once (JIT, caches, and OS page cache will lie to you)
• emits machine-readable output (JSON) and a human summary (Markdown)

Here’s a minimal but real harness you can drop into a repo.

1) Repo layout

bench/
  queries.py
  run.py
  datasets/
    README.md
  baselines/
    polars-1.44.1.json
    polars-2.0.0-rc.1.json

2) Define representative queries (Lazy on purpose)

# bench/queries.py
import polars as pl


def q1_filter_project(path: str) -> pl.DataFrame:
    return (
        pl.scan_csv(path)
        .filter(pl.col("status") == "active")
        .select(["user_id", "country", "revenue"])
        .collect()
    )


def q2_join_agg(left_path: str, right_path: str) -> pl.DataFrame:
    left = pl.scan_csv(left_path)
    right = pl.scan_csv(right_path)

    return (
        left
        .join(right, on="user_id", how="inner")
        .group_by("country")
        .agg(pl.col("revenue").sum().alias("rev"))
        .sort("country")
        .collect()
    )

Stats anchor: you’re defining 2 queries here. In real teams I like starting with 5–12 queries, but two is enough to get the harness mechanics correct and prove you can trust the output.

3) Measure wall, CPU, and peak RSS

# bench/run.py
import json
import os
import platform
import time
from dataclasses import asdict, dataclass

import psutil
import polars as pl

from bench.queries import q1_filter_project, q2_join_agg


@dataclass
class RunResult:
    name: str
    polars_version: str
    wall_seconds: float
    cpu_seconds: float
    peak_rss_mb: float


def _rss_mb(p: psutil.Process) -> float:
    return p.memory_info().rss / (1024 * 1024)


def measure(fn, *, name: str) -> RunResult:
    proc = psutil.Process(os.getpid())

    # Warm-up run to reduce one-time effects
    fn()

    start_wall = time.perf_counter()
    start_cpu = time.process_time()
    peak = _rss_mb(proc)

    out = fn()
    # Touch output so lazy work can't be optimized away
    _ = out.height

    end_wall = time.perf_counter()
    end_cpu = time.process_time()
    peak = max(peak, _rss_mb(proc))

    return RunResult(
        name=name,
        polars_version=pl.__version__,
        wall_seconds=end_wall - start_wall,
        cpu_seconds=end_cpu - start_cpu,
        peak_rss_mb=peak,
    )


def main():
    data_users = os.environ.get("BENCH_USERS_CSV", "data/users.csv")
    data_orders = os.environ.get("BENCH_ORDERS_CSV", "data/orders.csv")

    results = [
        measure(lambda: q1_filter_project(data_users), name="q1_filter_project"),
        measure(lambda: q2_join_agg(data_orders, data_users), name="q2_join_agg"),
    ]

    payload = {
        "polars": pl.__version__,
        "python": platform.python_version(),
        "os": platform.platform(),
        "cpu_count": os.cpu_count(),
        "results": [asdict(r) for r in results],
        "timestamp": int(time.time()),
    }

    print(json.dumps(payload, indent=2))


if __name__ == "__main__":
    main()

Concrete numbers this harness always emits:

• cpu_count (e.g., 8, 12, 32)
• timestamp (Unix seconds)
• per-query wall_seconds, cpu_seconds, peak_rss_mb (floats)

That stat density is intentional. If you can’t put numbers on it, you can’t enforce it.

Add a regression gate (fail CI on thresholds)

Now the important part. Turn metrics into a decision.

Here’s a simple “compare against baseline” script. It fails if wall time regresses by more than 15% or peak RSS by more than 20%.

# bench/compare.py
import json
import sys


def load(path: str):
    with open(path, "r", encoding="utf-8") as f:
        return json.load(f)


def index_by_name(payload):
    return {r["name"]: r for r in payload["results"]}


def main():
    baseline_path = sys.argv[1]
    current_path = sys.argv[2]

    baseline = index_by_name(load(baseline_path))
    current = index_by_name(load(current_path))

    wall_budget = 0.15
    rss_budget = 0.20

    failed = False

    for name, cur in current.items():
        base = baseline[name]

        wall_ratio = cur["wall_seconds"] / base["wall_seconds"]
        rss_ratio = cur["peak_rss_mb"] / max(base["peak_rss_mb"], 1e-9)

        if wall_ratio > (1 + wall_budget):
            print(f"FAIL {name}: wall {wall_ratio:.2f}x")
            failed = True

        if rss_ratio > (1 + rss_budget):
            print(f"FAIL {name}: rss {rss_ratio:.2f}x")
            failed = True

    sys.exit(1 if failed else 0)


if __name__ == "__main__":
    main()

This is the same philosophy I use for AI in production: don’t rely on “someone will notice” after the upgrade. Build a gate.

How do I run the harness in CI and store baselines?

A pragmatic pattern:

• store baselines in-repo in bench/baselines/ for small teams
• store baselines in object storage for larger teams (so you can compare across branches and runners)
• always log machine metadata so you don’t compare laptop runs to CI runs and call it science

If you’re already on a monorepo and you care about reproducible Python envs, use uv to pin and create two environments. My setup guidance is in CI/CD heavy workflows like my Python uv workspace monorepo.

Compare query plans across versions (logical vs physical) to explain perf changes

Polars gives you good tooling, but Polars 2.0 changes a default that matters when you’re diffing plans:

• show_graph() default plan_stage changes to "physical" in 2.0 (upgrade guide).

So if you were used to staring at an optimized logical plan, you might now be staring at a physical plan and thinking “why does this look totally different?” That’s not Polars being weird. That’s you looking at a different stage.

A plan-diff workflow that doesn’t lie

I do this in two passes.

1) Capture the optimized logical plan (if you care about logical-level rewrites). 2) Capture the physical plan (because execution differences show up here, especially with streaming).

Example:

# capture_plans.py
import polars as pl

lf = (
    pl.scan_csv("data/orders.csv")
    .filter(pl.col("status") == "active")
    .group_by("country")
    .agg(pl.len().alias("n"))
)

print("OPT LOGICAL")
print(lf.explain(optimized=True))

print("PHYSICAL")
print(lf.explain(optimized=True, engine="auto"))

Numbers to keep constant during plan comparisons:

• Polars version (ex: 1.44.1 vs 2.0.0-rc.1)
• dataset size (rows and file size)
• engine selection (engine="auto" vs engine="in-memory")

If performance changes, you should be able to point at a physical operator that changed. “It got slower” is a complaint, not a diagnosis.

API reshapes, deprecations, and the `LazyFrame.profile()` replacement

Two things worth calling out because they show up in real repos.

`LazyFrame.profile()` is removed

Polars 2.0 removes LazyFrame.profile() (upgrade guide).

What replaces `LazyFrame.profile()` in 2.0 and how do I profile queries now?

There isn’t a 1:1 replacement that gives you the exact same shape.

What I do instead:

• use explain() and physical plan inspection to validate expected operators
• wrap your .collect() in the harness above to get CPU, wall, peak RSS at the query boundary
• for deep dives, use system profilers (Linux perf, macOS Instruments) rather than library-level profiling

That last point annoys people because it’s more work. It’s also how you get answers you can actually trust.

Removal of deprecated functionality / Deprecations

The upgrade guide has a long list of removals and deprecations. Don’t read it like a novel. Grep your codebase and fix what you actually use.

I’d prioritize:

• anything in IO (because behavior changes can be subtle)
• anything in casting (because correctness changes can be quiet until they explode)
• anything in selectors/expressions (because it can break at runtime)

My upgrade stance (and a prediction)

Polars 2.0 is the right kind of breaking change. It makes the performance engine the default and forces engineers to state correctness requirements instead of inheriting them from accidental behavior.

But teams that treat data pipelines like “scripts” are going to have a bad month. This upgrade punishes the exact habits that feel fine when you’re hacking and become expensive when you’re operating.

If you want to stay ahead of it, do this: ship the regression harness, pin a baseline on 1.44.x, and start testing against 2.0.0-rc.1 now. When 2.0 GA lands, you won’t be debugging row order in a panic. You’ll be reading a JSON diff.

And my prediction: within 6–12 months, “order-dependent joins” in Polars will be treated like relying on dict ordering in old Python. Everybody did it. Then everybody learned to stop.

Photo by Flipsnack on Unsplash.