Anthropic CEO Urges Slower AI Race as Safety Risks Mount
Anthropic, founded in 2021 by former OpenAI researchers including Dario Amodei, has positioned its Claude models around safety and a Responsible Scaling Policy. The debate has become more urgent as frontier systems gain the ability to help build their successors, raising the prospect of recursive self-improvement. The stakes extend beyond competition among AI labs to cybersecurity, biological misuse and economic disruption, while any slowdown must also account for the strategic technology contest between the United States and China.
In an essay published Sept. 12, 2026, Amodei said AI capabilities had accelerated sharply since the summer and warned that within six to 12 months a misaligned agent swarm might seize much of the internet through a persistent botnet, potentially causing hundreds of billions of dollars in damage. He proposed embedded third-party evaluators, coordination among democracies and global coordination. Anthropic will unilaterally offer evaluators employee-like access; OpenAI CEO Sam Altman backed that step, while Elon Musk said Amodei was right.
All Coverage
3 original reportsThe Backstory
The history behind this eventAnthropic CEO Pushes Back on AI Power-Concentration Claim
The rapid advance of frontier AI has sharpened a fundamental governance dispute: whether powerful systems require centralized oversight to contain catastrophic risks, or a more plural market in which competing models and institutions check one another. Investor Gavin Baker argued that Anthropic CEO Dario Amodei’s worldview could lead to a future dominated by “Anthropic and the government,” raising concerns that safety-driven regulation may also concentrate commercial and political power.
Amodei responded directly in the latest exchange, rejecting the characterization that he favors an AI order controlled by Anthropic and governments and clarifying the company’s approach to safety and governance. The debate disclosed no transaction value, new funding figure or formal regulatory deadline. Its significance instead lies in the widening strategic split between tighter controls on frontier models and a decentralized framework built around multiple developers, competitive pressure and institutional checks.
Anthropic Calls for Global Pause in Frontier AI Development, Warns of Recursive Self-Improvement Risks
Recursive self-improvement, or RSI, refers to AI helping design and train more capable successor models, creating a cycle of accelerating capabilities. Anthropic argues that model failures could become more frequent and harder to understand if competition among frontier labs outpaces safety governance and society’s ability to adapt. It says the world must first establish verifiable safeguards that allow developers to slow or pause in coordination.
Anthropic published “When AI Builds Itself” on June 4, 2026, saying Claude had written more than 80% of the code merged at the company as of May that year. Engineers were merging eight times as much code per quarter as they did during 2021–2025, while the success rate on the most difficult and ambiguously specified coding tasks had risen to 76% within six months. The company said human review had become a bottleneck and that it would push governments and frontier AI developers to coordinate mechanisms for temporarily slowing development.
Subscribe to Mark Radar Weekly
Every Friday, the week's strongest signals in your inbox. Unsubscribe anytime.
If you search news on Google, you can set Mark Radar as a preferred source—our coverage will show up more often in your results. Set as preferred source on Google →