| Measuring the sloppiness of code(earendil.com) | |
| 273 points by doppp 1 day ago | 231 comments | |
tl;dr: LLM-generated code is often formally correct but "sloppy"—verbose, duplicated, or overly complex—and measuring this is hard because LLM-as-judge approaches are unreliable and human review doesn't scale. Using metrics from SlopCodeBench (verbosity and erosion based on cyclomatic complexity), the author found agent-generated code is roughly twice as verbose and eroded as human-written code in established repos. On iterative benchmarks where context resets between rounds, even state-of-the-art models score 0% strict pass rate, suggesting agents can't clean up their own accumulated slop. | |
HN Discussion:
| |