Researchers Urge Treating AI Agents as Untrusted Systems to Bolster Security

AI agents can autonomously access browsers, APIs, memory and execution tools. After a prompt-injection attack, the risk can escalate from an incorrect response to data leakage or unauthorized actions. Researchers from Google, the University of California San Diego, the University of Wisconsin–Madison and Cornell University argue that model robustness alone is insufficient. Models should instead be treated as untrusted components, with external systems enforcing security boundaries.

The paper, “Agent Security is a Systems Problem,” was submitted to arXiv on May 18, 2026, and updated on May 20. The authors reviewed 11 real-world attacks and found that all violated secure information-flow principles, while most also breached the principle of least privilege. They proposed three implementation priorities: separating instructions from data, using verifiable least-privilege sandboxes, and enforcing information-flow controls. The research did not involve trading or cite any monetary amounts.