Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA(fireworks.ai)
727 points by piotrgrabowski 14 hours ago | 384 comments
tl;dr: Benchmarking Kimi K3 (open) against Fable 5 (closed) across ~1,030 agentic tasks shows the two models perform comparably overall but specialize in different domains—K3 wins on terminal/security/crypto work, Fable on multi-language coding and data viz. Oracle routing between them achieves 93% accuracy while sending 72-96% of traffic to the cheaper K3, yielding up to 50x cost savings versus Fable alone. The takeaway: use a cost-optimized open model as default and route to premium models only for the long tail.
HN Discussion:
  • Benchmarks are gamed and Fireworks has commercial incentive to promote K3
  • The benchmark is self-promotion for Fireworks' router product
  • Explains/summarizes the routing methodology and findings
  • Questions whether routing is practical for individual users due to cache and task-switching costs
  • Asks why K3 specifically, questioning if other cheap models would work equally well