Beating GPT-5.6 Sol on retrieval with 100x cheaper open models(neon.com)
352 points by moonikakiss 19 hours ago | 85 comments
tl;dr: Castform uses RL post-training to make small open-source models competitive with frontier models like GPT-5 on agentic retrieval tasks, claiming ~100x cost reduction. The pipeline runs on Neon's Lakebase Postgres, which handles synthetic training data generation, hybrid (BM25 + vector) search during RL rollouts, and autoscaling for bursty training workloads. The pitch: turn your existing corpus into training tasks with a reward function (retrieval + citation + correctness), skipping the usual data engineering and ML infra overhead.
HN Discussion:
  • Specialized purpose-built models for specific tasks like retrieval are a promising direction
  • Frontier labs' business model is threatened by cheaper commoditized specialized models
  • Missing comparisons to other cheaper models (Luna, DSFlash) and standard retrieval benchmarks undermine claims
  • ~Cloud-based service is unusable for sensitive data; need self-hostable option
  • The real problem is retrieval architecture (chunking/RAG) rather than model training