AI Tool Forecasts Lung Cancer Risk From a Single CT Scan

An artificial intelligence (AI) tool named “Sybil” was able to forecast both short- and long-term lung cancer risk based on a single low-dose CT (LDCT) scan, researchers reported.

The AI tool was trained and validated on LDCTs from the National Lung Screening Trial (NLST), according to Lecia Sequist, MD, MPH, of Harvard Medical School and Massachusetts General Hospital (MGH) in Boston, and colleagues.

Sybil was able to predict lung cancer within 1 year with an area under the curve (AUC) of 0.92 (95% CI 0.88-0.95), and a 2-year AUC of 0.86 (95% CI 0.82-0.90), on an NLST test set, as well as a concordance index over 6 years of prediction of 0.75 (95% CI 0.72-0.78), they detailed in the Journal of Clinical Oncology.

Additionally, Sybil maintained its performance across sex, age, and smoking history subgroups, with the results validated in two large separate datasets totalling more than 23,000 LDCT screens.

In an accompanying editorial, Gerard Silvestri, MD, MS, of the Medical University South Carolina in Charleston, and James R. Jett, MD, of National Jewish Health in Denver, highlighted the “lengths and depths” the investigators went to get their model “right,” and called the model’s overall performance “outstanding.”

They noted that the predictive tool offered several advantages from a “practical perspective.” For example, it doesn’t require patient demographics, risk factors or manual identification, and characterization of nodules, “each of which take time and expertise, limiting the practical use of some of the other [deep learning models],” they wrote.

Silvestri and Jett also suggested the model could decrease unnecessary work-ups and invasive testing in nodules that aren’t predicted to lead to a future cancer, as well as predict which patients could safely lengthen screening intervals, or even discontinue screening entirely if they have an extremely low risk of developing cancer.

“The model might be able to identify at-risk groups that currently do not meet U.S. Preventive Services Task Force criteria for inclusion in a screening program,” they added. “This is an intriguing possibility because nearly half the lung cancers diagnosed in the United States currently do not qualify for screening.”

In developing Sybil, the investigators accessed radiologic and clinical data from a sample of 15,000 NLST participants in the LDCT arm who were split into training, development, and test sets. For purposes of testing, Sequist and colleagues noted that Sybil’s input was limited to LDCT images only, and that no image annotation or clinical information was provided.

In order to externally validate Sybil, they retrospectively obtained 13,309 LDCTs from 6,392 adults who underwent lung cancer screening at MGH from 2015 to 2021, and 12,480 LDCTs from 10,696 adults who had undergone LDCTs for lung cancer screening at Chang Gung Memorial Hospital (CGMH) in Taoyuan, Taiwan from 2007 to 2019.

The CGMH population differed from that of the NLST and MGH groups in that any adult without a personal cancer history could obtain an LDCT, even without a history of smoking, the authors noted.

Sequist’s group found that Sybil’s performance for these two independent test sets was comparable to that demonstrated with the NLST set, with an AUC for lung cancer prediction at 1 year of 0.86 (95% CI 0.82-0.90) for the MGH dataset and 0.94 (95% CI 0.91-1.00) for the CGMH dataset. Concordance indices over 6 years were 0.81 (95% CI 0.77-0.85) and 0.80 (95% CI 0.75-0.86), respectively.

They reported the performance of the model decreased when they removed identifiable pulmonary nodules known to be cancerous from the analysis set for an AUC at 2 years of 0.81 (95% CI 0.74-0.86) and a 6-year AUC of 0.69 (95% CI 0.63-0.74), but that Sybil “still possessed predictive power.”

Sequist and colleagues also compared Sybil’s false-positive rate, and found it reduced the rate to 8% for baseline scans in the NLST set versus 14% for Lung-RADS 1.0, while maintaining equivalent sensitivity.

Study limitations included its retrospective nature and the fact that the investigators did not have “detailed smoking data from CGMH subjects, so conclusions about Sybil’s ability to predict lung cancer from images in nonsmokers remain speculative.”

Sequist and colleagues pointed out that before Sybil can be studied prospectively, it should be determined whether it is generalizable. For instance, NLST screening scans were obtained from patients who were overwhelmingly white (92%) so “none of the cohorts presented here include sufficient Black or Hispanic patients to have confidence in broad applicability yet.” Also, changes in CT technology since the NLST “might adversely affect Sybil’s translation.”

The Sybil algorithm is publicly available, including the NLST dataset and image annotations.