Tracking down the 16-year-old WAL-reset SQLite bug(tailscale.com)
1199 points by ropbear 9 days ago | 232 comments
tl;dr: Tailscale spent six months chasing intermittent SQLite database corruption across their control plane shards, working with SQLite's core developers to eventually identify a 16-year-old data race between checkpoints and write transactions (dubbed the "WAL-Reset bug"). Their aggressive manual checkpointing—an unusual but supported configuration—made them uniquely likely to trigger it. The fix shipped in SQLite 3.51.3, and Tailscale later confirmed via instrumentation that the race condition was indeed occurring in production.
HN Discussion:
  • Praise for Tailscale funding open source tooling and engaging SQLite's commercial support
  • Admiration for SQLite's engineering rigor and reputation for reliability
  • ~SQLite's testing methodology has limits and modern deterministic concurrency testing would catch such bugs better
  • Curiosity or mild critique about Tailscale's unusual aggressive checkpointing decision that triggered the bug
  • Reflection on the industry losing tribal knowledge from experts about non-standard database usage risks