Artificial intelligence is delivering health guidance at an unprecedented scale. Every week, more than 300 million people ask ChatGPT health questions. Now, the federal government is moving AI deeper into medical care to communicate directly and autonomously with patients. Chris Klomp, the nominee for deputy secretary of Health and Human Services, has called AI in patient care the agency’s “holy grail” for improving the nation’s health.
Government and tech leaders believe AI will benefit us because it is remarkably good at retrieving factual medical information. Although nearly all published safety evaluations focus on accuracy, it is only one dimension of safety. Measuring accuracy cannot detect a subtle form of harm: Even a factually accurate health message can change behavior in ways that make people less healthy.
