Study Warns Medical AI Models Could Open New Front in Patient-Record Privacy Breaches

AI systems used for medical diagnosis are often trained on patient records stripped of identifiers such as names and national identification numbers, but the models may still retain characteristics of individual records. A German research team said “membership inference attacks” can analyze a model’s output to determine whether a particular patient’s data was included in its training set, challenging existing de-identification and privacy-review mechanisms.

The latest study found that membership inference attacks on medical classification models identified training data with near-perfect accuracy and produced virtually no false positives at the individual level. Publicly available information did not identify the research institution, publication date or research funding. The team is primarily calling on the industry to revise privacy-review standards and introduce mathematical safeguards such as differential privacy.