Why Software Factories Fail (or: harness engineering is not enough)(github.com)
373 points by dhorthy 1 day ago | 263 comments
tl;dr: Lights-off "software factories" where AI agents ship code without human review fail because current models degrade codebase maintainability over time — a problem RL training can't fix, since benchmarks like SWE-bench reward passing tests but impose no penalty for bad design, and there's no fast oracle for good architecture. The author argues teams should stop chasing 10-100x speedups and instead front-load human judgment via product specs, system architecture, program design, and vertical slices, realistically achieving 2-3x faster shipping while keeping quality intact.
HN Discussion:
  • Humans must define intent and understand code; AI can implement but not comprehend
  • Author lacks credibility and provides no evidence for his claims
  • Article's July 2025 experiment predates recent model improvements, making conclusions outdated
  • Architecture quality is subjective/aesthetic, aligning with article's concerns about oracles
  • Rigorous upfront specifications (RFCs, specs) make AI implementation viable, echoing front-loading judgment