Is AI reasoning right for the wrong reasons?(quantamagazine.org)
207 points by retupmoc01 1 day ago | 235 comments
tl;dr: Large reasoning models (LRMs) produce impressive results—solving open math problems and winning IMO gold—but growing research shows their "chains of thought" often aren't faithful representations of internal reasoning: irrelevant or filler tokens work just as well, and 30-60% of "thinking steps" have minimal causal impact on outputs. Researchers like Subbarao Kambhampati argue LRMs are doing "approximate retrieval" rather than genuine step-by-step reasoning, with intermediate tokens serving to prime the model rather than narrate actual thought. The debate matters because trusting AI in non-verifiable domains requires knowing whether models are right for the right reasons.
HN Discussion:
  • Debate is semantic navel-gazing; what matters is functionality, not defining 'reasoning'
  • Classifiers/LLMs being right for wrong reasons is a well-known, expected phenomenon
  • Reasoning tokens are a mechanical way to emulate deeper computation/recursion, not genuine thought
  • AI terminology is misleading marketing; 'reasoning' is wishful mnemonics obscuring what's happening
  • ~Nobody actually understands what's happening inside these models