Opinion | Hey Siri: Is Health Info From Voice Assistants Accurate?

All voice assistants are not created equal, at least not when it comes to providing their users with helpful and correct medical information.

Cases of colorectal cancer have been steadily rising in younger adults, including those in their 30s and 40s, and in our practice people in these age groups are more likely to mention medical information found on their smartphones before their appointment. Young adults are also less likely to utilize the healthcare system.

Our team set up an experiment to evaluate the accuracy of voice assistants in sharing medical information and to assess the credibility of the sources they point to when users seek basic information about colorectal cancer screening. Our results, presented at Digestive Disease Week 2022 in May, were both reassuring and disconcerting, depending on which assistant you ask.

Using simple first-person language, we asked the most popular voice assistants — Apple’s Siri, Google Assistant, Amazon Alexa, and Microsoft’s Cortana — five of the most common questions we get from patients in primary care and in our gastroenterology clinic:

• When should I start getting tested for colon cancer?
• How often should I get tested for colon cancer?
• What are the different tests for colon cancer?
• What could I do to reduce my risk for colon cancer?
• What are the symptoms for colon cancer?

First the good news: We found that Siri was the most medically accurate, getting five out of five right. The app provided relevant information and linked to the credible sources that we would use as clinicians, such as the CDC, U.S. Preventive Services Task Force, and the major gastroenterological societies.

The bad news: While Google Assistant also provided correct answers to all five questions, it placed advertisements at the top of the search results, forcing users to scroll through potentially biased commercial information to get to the most objective responses.

Amazon Alexa responded correctly to three of the five questions, and Cortana gave accurate information to only two questions. These lower-performing apps failed to provide clear and accurate information about steps to reduce the risk of colon cancer. They also provided vague and misleading descriptions of symptoms that many people experience for other reasons, potentially evoking unnecessary fear.

While the poor results from Alexa and Cortana are troubling, the biggest red flag we saw in this study was Google Assistant’s prominent presentation of ads. We understand that Google makes its money through targeted advertising, but when serious health issues are at stake, advertising should not be the priority.

If someone with a serious health problem is seeking help from a voice assistant, advertisements for a product that may not help could pop up, instead of the person being directed to promptly seek medical care. For example, Google Assistant advertised Cologuard, a commercial product that screens for colorectal cancer, well before they recommended a screening colonoscopy for someone over the age of 45. Google should be more responsible and prioritize unbiased search results.

This was a small pilot study, but even this initial experiment provides valuable insights that can help clinicians in conversations with their patients. We have a duty to explore how patients are getting information and what prompted them to come to us. We should also know what’s happening in the world outside of the scope of our clinical practice. When “Black Panther” star Chadwick Boseman was diagnosed with late-stage colorectal cancer and died in 2020 at age 43, this led to a surge in searches for colon cancer and an uptick in young, frightened patients coming to our clinics. Understanding where this fear was coming from sparked important conversations with patients coming into the clinic and created an opportunity for shared decision-making conversations about how we can screen them and treat them going forward.

We hope to expand this study in multiple ways. We want to add more questions and also have laypersons pose questions in their own words to ensure a real-world experience. We also want to expand the study to ask about functional diseases, such as irritable bowel syndrome. Patients are likely to ask voice assistants about functional diseases, yet responses to such questions are less clearcut than questions about colorectal cancer screening, which has specific guidelines.

Voice assistants are doing an adequate job, but there’s clearly room for improvement. Voice assistants should prioritize peer-reviewed and respected resources when providing medical advice. Unfortunately, none of the four voice assistants directed their user to speak to a doctor. With the right algorithms and fine tuning, these devices can assist patients and doctors alike.

Atul Sinha, MD, is a resident physician at Nassau University Medical Center in New York.