| A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost(hightouch.com) | |
| 572 points by theanomaly 5 days ago | 146 comments | |
tl;dr: Testing LLMs for grading identity resolution over-merges, Hightouch found that cheap "decision models" (Jev, GPT-6 Luna) hit 95-97% accuracy versus 99% for Claude Opus 5.5, but at 40-90x lower cost. A hybrid approach—running Jev first and escalating low-confidence cases (30-90%) to Opus—matches Opus's accuracy at ~4x lower cost. Jev beats Luna in the hybrid setup despite worse standalone performance because its confidence scores are better calibrated, while Luna is overconfident on wrong answers and fails to escalate hard cases. | |
HN Discussion:
| |