AI Advice Makes People Less Likely to Admit Ignorance, Study Finds

Generative AI is moving rapidly into search, education and workplace decision-making, raising concern that fluent machine output may be mistaken for genuine understanding. Researchers at the University of Milano-Bicocca, École Normale Supérieure and Sapienza University of Rome tested whether AI advice changes users’ metacognitive threshold for answering difficult questions. The issue matters because recognizing when evidence is insufficient is central to sound judgment, especially as AI assistants become routine.

In a preprint posted to arXiv on July 15, the team reported five experiments involving 3,132 participants. The questions were designed so the AI advice was wrong. With AI available, the share choosing “I don’t know” fell to 3% from 44%, while accuracy dropped to 9% from 27%. Confidence nevertheless jumped to 76% from 30%. When correct answers were rewarded and wrong ones penalized, abstention recovered only to 8% and accuracy to 16%, reinforcing calls for education and AI-literacy policies.