Artificial Intelligence Rarely Passes Radiologists’ Mock Exams

TUESDAY, Dec. 27, 2022 (HealthDay News) — An artificial intelligence candidate can pass two of 10 mock Fellowship of the Royal College of Radiologists (FRCR) examinations when noninterpretable images are excluded from analysis, according to a study published online Dec. 21 in The BMJ.

Susan Cheng Shelmerdine, M.B.B.S., from the Great Ormond Street Hospital for Children in London, and colleagues conducted a prospective multireader diagnostic accuracy study involving one artificial intelligence candidate and 26 radiologists who had passed the FRCR examination in the previous 12 months.

The researchers found that the artificial intelligence candidate achieved an average overall accuracy of 79.5 percent and passed two of 10 mock FRCR examinations when noninterpretable images were excluded. The average radiologist achieved 84.8 percent average accuracy and passed four of 10 mock examinations. For the artificial intelligence candidate, the sensitivity and specificity were 83.6 and 75.2 percent, respectively, compared with summary estimates of 84.1 and 87.3 percent, respectively, across all radiologists. The artificial intelligence candidate was incorrect in 9 percent of radiographs that were correctly interpreted by >90 percent of radiologists. Most imaging pitfalls related to interpretation of musculoskeletal rather than chest radiographs.

“Further training and revision are strongly recommended, particularly for cases the artificial intelligence considers ‘non-interpretable,’ such as abdominal radiographs and those of the axial skeleton,” the authors write. “Increased familiarity with less common and more subtle bony pathologies will also help to boost the chances of examination success.”