Making Postgres 300x faster for analytics: batching, operator fusion, and SIMD(malisper.me)
326 points by poly2it 14 days ago | 163 comments
tl;dr: pgrust v0.2 claims 300x faster analytics than Postgres (beating ClickHouse on ClickBench) by rewriting the query engine to overcome Postgres's row-at-a-time Volcano model. The post demonstrates three key optimizations on a SUM query: batching (processing 1024 rows at a time with stack-allocated buffers), operator fusion (combining scan and aggregate into one node to eliminate copies), and SIMD (parallel float operations that compilers won't auto-generate due to associativity). Combined, these took a toy query engine from 1.3s to 135ms—10x faster—with JIT compilation teased for a follow-up.
HN Discussion:
  • Author engagement addressing anticipated trust/correctness concerns about the project
  • Postgres's ecosystem, trust, and longevity matter more than raw performance gains
  • Project appears immature, buggy, or misleadingly presented as a serious team effort
  • Excitement that this demonstrates viability of adaptive planning and modern query engine techniques
  • Curious technical questions about architecture details like IO/thread scheduling and window functions