Weak AI Rules Can Undermine Safety, Study Finds

AI products are typically built across a supply chain: general-purpose model developers establish baseline capabilities and safeguards, while downstream specialists adapt the systems for uses such as customer-service chatbots or medical diagnosis. That division makes regulation consequential beyond the company directly covered. Researchers at Cornell University and Carnegie Mellon University used economic theory and game theory to examine how safety mandates reshape incentives for both upstream creators and downstream firms.

The study, “The Backfiring Effect of Weak AI Safety Regulation,” was published in the Proceedings of the National Academy of Sciences on July 20, 2026. Across 49,686 simulated games, the model found that a low safety floor aimed mainly at downstream firms could encourage general-purpose providers to free-ride and cut their own safeguards, producing less safety than no regulation. Properly calibrated requirements on both sides could improve safety, performance and profits, though the authors said real-world evidence is still needed.