UIUC Researchers Build PlainQAFact to Test AI Summary Factuality

Large language models can make biomedical research more accessible by rewriting technical papers for general readers. The process, however, often adds definitions, background or examples that do not appear in the source abstract, leaving conventional entailment- and question-answering-based metrics unable to verify the extra material. Researchers at the University of Illinois Urbana-Champaign’s School of Information Sciences developed PlainQAFact to identify such unsupported additions and reduce hallucination risks in health communication.

Zhiwen You and Yue Guo submitted the preprint on March 11, 2025, with the study later published in the Journal of Biomedical Informatics in 2026. PlainQAFact first classifies sentences as source simplifications or elaborative explanations, then retrieves external medical knowledge only for the latter before applying question-answering checks. Its PlainFact benchmark contains 2,740 expert-annotated sentences; 44% were elaborative, and 66% of those could not be verified directly from the original abstract. Tests against five widely used metrics across three datasets showed stronger overall factual-consistency performance.