Study Finds Nearly Half of AI Chatbots’ Medical Advice Problematic as OpenAI and Anthropic Expand in Healthcare

People are increasingly turning to generative AI for information before seeking medical care, although general-purpose chatbots are not approved to provide medical diagnoses. A study published in BMJ Open and partly funded by the Center for Artificial Intelligence Research at Wake Forest University School of Medicine tested five major products. It highlighted the risk that incorrect advice and fabricated citations could delay diagnosis and treatment.

The study, published by BMJ Open on April 14, tested five chatbots with 50 questions in February 2025. Of 250 responses, 49.6% were rated problematic, while Grok produced 29 responses classified as highly problematic. OpenAI launched healthcare features on January 7, and Anthropic introduced Claude for Healthcare on January 11. OpenAI then acquired Torch on January 12 for a reported roughly $100 million.