Former Anthropic Researcher Quits, Warns of Superintelligence Risk
Leading artificial-intelligence companies are racing to build increasingly capable systems, including models that may eventually improve their own performance. That prospect has made alignment, control and independent oversight central industry concerns. Jacob Coxon, a researcher who worked at both OpenAI and Anthropic, argues that a rapid leap toward superintelligence could outpace existing safeguards, leaving humans unable to reliably direct or contain the technology.
Coxon recently said he had resigned from Anthropic and was leaving the AI industry, warning that competitive pressure was pushing leading technology companies to pursue powerful systems without accepting commensurate responsibility for safety. He said self-improving superintelligence could pose a catastrophic threat to humanity within the next decade. The timeline reflects Coxon’s assessment rather than an established forecast, but his departure has renewed scrutiny of corporate safety commitments and the adequacy of AI regulation.
All Coverage
2 original reportsThe Backstory
The history behind this eventAI Insiders Urge US to Create Frontier AI ‘Brake Pedal’
Employees and experts affiliated with OpenAI, Anthropic, Google and other leading AI labs are pressing Washington to address a widening gap between frontier-model development and government oversight. Their concern centers on the prospect that AI systems could automate increasingly large parts of AI research itself, sharply accelerating capability gains and leaving regulators little time to respond. The signatories argue that without enforceable safeguards and international coordination, competitive pressure could push companies and countries to keep scaling systems even as risks of misuse or loss of control grow.
As of July 30, 2026, more than 1,100 AI insiders had backed a joint statement urging the US government to create a “brake pedal” for frontier AI or move faster on globally coordinated guardrails. The proposal calls for mechanisms that could slow advanced-model development when agreed risk thresholds are reached, rather than a blanket halt to AI research. It also asks policymakers to put effective oversight and international agreements in place before automated AI research can drive a rapid, self-reinforcing surge in technical progress.
Subscribe to Mark Radar Weekly
Every Friday, the week's strongest signals in your inbox. Unsubscribe anytime.
If you search news on Google, you can set Mark Radar as a preferred source—our coverage will show up more often in your results. Set as preferred source on Google →