ChatGPT Offers Evidence-Based Answers for Common Public Health Questions

FRIDAY, June 9, 2023 (HealthDay News) — ChatGPT consistently provides evidence-based answers to public health questions, according to a research letter published online June 7 in JAMA Network Open.

John W. Ayers, Ph.D., from the Qualcomm Institute at the University of California San Diego in La Jolla, and colleagues assessed how well ChatGPT handles general health inquiries from the lay public. Responses to 23 questions grouped from categories (addiction, interpersonal violence, mental health, and physical health) were evaluated by two independent reviewers.

The researchers found that ChatGPT responses were a median of 225 words, with a reading level that ranged from 9th grade to 16th grade. Twenty-one of the 23 responses were determined to be evidence-based, but only five responses made referrals to specific resources (two related to addiction, two for interpersonal violence, and one for mental health).

“Although search engines sometimes highlight specific search results relevant to health, many resources remain underpromoted,” the authors write. “Artificial intelligence assistants may have a greater responsibility to provide actionable information, given their single-response design. Partnerships between public health agencies and artificial intelligence companies must be established to promote public health resources with demonstrated effectiveness.”

Several authors disclosed financial ties to the medical technology industry.