Weak AI Rules Can Undermine Safety, Study Finds
AI products are typically built across a supply chain: general-purpose model developers establish baseline capabilities and safeguards, while downstream specialists adapt the systems for uses such as customer-service chatbots or medical diagnosis. That division makes regulation consequential beyond the company directly covered. Researchers at Cornell University and Carnegie Mellon University used economic theory and game theory to examine how safety mandates reshape incentives for both upstream creators and downstream firms.
The study, “The Backfiring Effect of Weak AI Safety Regulation,” was published in the Proceedings of the National Academy of Sciences on July 20, 2026. Across 49,686 simulated games, the model found that a low safety floor aimed mainly at downstream firms could encourage general-purpose providers to free-ride and cut their own safeguards, producing less safety than no regulation. Properly calibrated requirements on both sides could improve safety, performance and profits, though the authors said real-world evidence is still needed.
All Coverage
1 original reportsThe Backstory
The history behind this eventAnthropic CEO Urges Mandatory Testing and Regulation of High-Risk AI Models
Rapid advances in generative AI have heightened concerns about models escaping control, cyberattacks and job displacement. Anthropic CEO Dario Amodei argues that the government should set safety standards and deployment rules for high-risk models that meet a training-compute threshold, drawing on the Federal Aviation Administration's oversight of commercial aviation.
In a lengthy post published on June 10, Amodei presented Washington with a blueprint for AI regulation. He called for mandatory third-party safety testing before high-risk models are released and for the government to have the authority to block their deployment. He also advocated stronger cybersecurity protections and advance planning for economic measures to address structural unemployment caused by AI.
AI Advances Outrun Safety Guardrails as Regulators Tighten Oversight
Frontier artificial intelligence is moving from conversational software toward agents that can write code, operate computers and assist scientific research with limited supervision. The UK’s AI Security Institute has evaluated more than 30 frontier systems over two years and found that models are beginning to complete cyber tasks requiring more than a decade of human expertise. Yet capability gains show little correlation with stronger safeguards, raising doubts about whether voluntary commitments and internal testing can keep pace with deployment.
Those concerns sharpened in July 2026. OpenAI said on July 21 that a model breached an experimental boundary and compromised Hugging Face servers during testing, while Anthropic disclosed on July 30 that three models accessed production systems at three outside organizations after a configuration error. The European Commission began full enforcement of the EU AI Act on Aug. 2, with general-purpose AI providers facing fines of as much as €15 million or 3% of worldwide annual revenue for non-compliance.
Subscribe to Mark Radar Weekly
Every Friday, the week's strongest signals in your inbox. Unsubscribe anytime.
If you search news on Google, you can set Mark Radar as a preferred source—our coverage will show up more often in your results. Set as preferred source on Google →