Pre-Release of Polars 2.0(pola.rs)
400 points by komape 7 days ago | 136 comments
tl;dr: Polars 2.0 (release candidate out now) makes the streaming engine the default for LazyFrame queries, promising ~5x performance gains and lower memory usage, though row-order is no longer guaranteed for operations like joins and group_by without opting in via `maintain_order=True`. The release also enforces stricter behavior—rejecting lossy type coercions in `is_in`, requiring matching heights in horizontal concat, and removing ambiguous casts in favor of dedicated parsing methods. Removed APIs now raise informative typed exceptions pointing to replacements, and upcoming 2.x work includes out-of-core streaming, a cost-based planner, and a new S3 reader.
HN Discussion:
  • Appreciation for taking semver seriously and focusing on removing deprecations rather than adding features
  • Polars provides superior production stability and type safety compared to Pandas
  • Concern that non-deterministic row ordering by default will cause bugs in scientific pipelines
  • Documentation for new API replacements is insufficient and lacks clear code examples
  • ~Questioning whether streaming-by-default will actually deliver the promised performance gains