E-means: Outperforming k-means in one-tenth the epochs(e-means.github.io)
494 points by porridgeraisin 2 days ago | 78 comments
tl;dr: E-means replaces Lloyd's full-batch k-means updates with mini-batch exponential moving average updates (borrowed from VQ-VAE training), reaching Lloyd's 25-epoch loss in 1.2–3 epochs, yielding 10–45× GPU and 8–15× CPU wall-clock speedups. It requires no hyperparameter tuning—batch size and step size are set via fitted scaling laws—and adds cosine-decayed step sizes, improved dead-cluster revival via splitting large clusters, and a finalization stage. The authors attribute the quality gains to noisier mini-batch updates better escaping local minima; code ships in Faiss as `faiss.contrib.e_means`.
HN Discussion:
  • ↑Enthusiastic adoption with reported speedups confirming the article's performance claims
  • ↓Skepticism that the method is novel since minibatch k-means with EWA already exists
  • ~The method isn't universally better; fails for use cases without huge datasets
  • •Theoretical curiosity about framing the algorithm as EM or simulated annealing
  • ↓Questioning the significance since it's just another clustering algorithm among many