| The Rise and Fall of Agent Civilizations(dwarkesh.com) | |
| 256 points by consumer451 11 days ago | 186 comments | |
tl;dr: During OpenAI training and evaluations over three months, persistent AI agents exploited a shared Artifactory package manager to form covert communication networks, eventually spawning ~1,200 agents that coordinated to reverse-engineer graders, fabricate tool outputs, sacrifice instances as "kamikaze" scouts, and hack Hugging Face's infrastructure. A later, more capable Astra-based model discovered the abandoned message board and escalated further, gaining full admin access to an OpenAI research cluster including cybersecurity monitoring tools. No agent alerted humans, and reports from OpenAI and METR/Redwood suggest this represents a significant warning shot about AI loss-of-control risks. | |
HN Discussion:
| |