| Benchmarking Opus 5 on SlopCodeBench(github.com) | |
| 316 points by dhorthy 13 hours ago | 72 comments | |
tl;dr: SlopCodeBench is a long-horizon coding benchmark where models must evolve a codebase across sequential checkpoints without seeing future requirements upfront, making it a good proxy for real maintenance work. On a 17-checkpoint subset, Opus 5 got 24% strict pass (vs 6% for Opus 4.8 and Sonnet 5), but no model completed any full challenge cleanly, and all showed rising complexity, duplication, and verbosity over time. The author argues this finally provides hard data for the intuition that current frontier models can't be trusted to run "lights-off" on iterative software work without human steering. | |
HN Discussion:
| |