Checking the Symptom Checkers

With further development, symptom checking programs may help save lives and reduce costs.Marco_Piunti / iStock / Thinkstock

More than a third of adults nationwide regularly use the Internet to help diagnose their ailments. They often start with online search engines, which can return confusing and unreliable information.

Recently, sophisticated programs called symptom checkers have emerged. They ask users a series of questions about their symptoms and analyze the responses using computerized algorithms. These programs offer a possible diagnosis and may also provide triage advice—whether users should seek care and, if so, at what kind of facility and how quickly. For people with a life-threatening problem, such as stroke or heart attack, symptom checkers may save lives by guiding them to seek emergency care. For those with a problem that doesn’t require a medical visit, the programs may provide reassurance and save both trouble and costs.

The impact of symptom checkers on health care depends on their accuracy. A team led by Dr. Ateev Mehrotra of Harvard Medical School and Beth Israel Deaconess Medical Center evaluated the performance of symptom checkers. Between June 2014 and November 2014, they searched for free symptom checkers online, in the Apple app store, and on Google Play. Many symptom checkers were found to use the same medical content and logic as others. For this and other reasons, the final set the scientists evaluated included 23 programs.

To evaluate the symptom checkers, the researchers used 45 standardized patient vignettes that included both common and uncommon conditions. The vignettes fell into 3 categories: 15 for which emergency care is needed, 15 for which non-emergency care is reasonable, and 15 for which self care is sufficient. The study was funded by NIH’s National Institute of Allergy and Infectious Diseases (NIAID). Results were published on July 8, 2015, in the BMJ.

The researchers found that the symptom checkers listed the correct diagnosis as the first option in 34% of cases and provided the correct diagnosis within the top 20 possible diagnoses 58% of the time. The checkers provided the appropriate triage advice in 57% of the evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% of emergency, 55% of non-emergency, and 33% of self-care cases. Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% to 78% of the patient evaluations. Overall, the checkers were risk adverse, encouraging users to seek care for conditions in which self care would be reasonable.

The researchers note that it’s often more important to know whether to seek emergency care than to know the exact diagnosis. Thus, their findings suggest that symptom checkers can be helpful. “These tools may be useful in patients who are trying to decide whether they should get to a doctor quickly. But in many cases, users should be cautious and not take the information they receive from online symptom checkers as gospel,” Mehrotra says.

“With symptom trackers, we're looking at the first generation of a new technology,” first author Dr. Hannah Semigran says. “It’s important to continue to track their performance to see if they can reach their full potential in helping patients get the right care.”

—by Harrison Wein, Ph.D.