Published
The 80th Korean Congress of Radiology (KCR 2024)
Authors
Taehee Kim1, Heejun Shin1 , Yongsub Song2 , Jonghyuk Lee3 , Hyungjin Kim3 , Dongmyung Shin1
Affiliations
1Radisen, Korea, Republic of
2Kim’s Eye Hospital, Korea, Republic of
3Seoul National University Hospital, Korea, Republic of
shinsae11@radisentech.co
*to be published soon.
Purpose
This study aimed to assess the performance of commercially available AI software in detecting lung cancer on chest radiographs from the Prostate, Lung, Colorectal, and Ovarian (PLCO) cancer screening trial and compare it to the performance of radiologists who participated in the trial.
Materials and Method
Chest radiographs from 77,443 participants in the PLCO trial were analyzed. Individuals with other cancers during the screening period or lung cancer more than a year after the screening period were excluded. The AI assessment focused on 25,988 radiographs of 24,370 individuals with a lung cancer incidence rate of 1.5%. A commercially available A I software (AXIR-CX; Radisen Co. Ltd.) providing image-level probability for the presen ce of nodules and masses was used. Chest radiographs with higher probabilities above a pre-specified threshold (= 0.8) of AI for nodule and mass were considered lung cancer -suspicious. The AI algorithm’s lung cancer detection performance was evaluated using sensitivity, specificity, positive prediction value (PPV), negative prediction value (NPV), a nd accuracy with reference to clinically confirmed lung cancer results. X-ray reading res ults (cancer-suspicious vs. non-suspicious) of radiologists who had participated in the sc reening trial were compared to the AI’s reading results. Statistical significance between t he reading results was determined using McNemar’s and Fisher’s exact tests.
Results
The AI software exhibited higher specificity (0.910 for AI vs. 0.803 for radiologists, p<0.0 01), PPV (0.054 for AI vs. 0.032 for radiologists, p<0.001), and accuracy (0.901 for AI v s. 0.797 for radiologists, p<0.001) than the radiologists but reported lower sensitivity (0.3 26 for AI vs. 0.412 for radiologists, p=0.001). When the sensitivity of the AI was adjusted to match that of the screening radiologists, the AI algorithm demonstrated higher specifi city (0.815 for AI vs. 0.803 for radiologists, p<0.001) and accuracy (0.809 for AI vs. 0.797 for radiologists, p<0.001).
Conclusion
The AI software displayed higher specificity, positive prediction value, and accuracy but l ower sensitivity than the screening radiologists. In addition, when the AI’s sensitivity was tuned to match that of the screening radiologists, it reported higher specificity and accur acy. Given its higher specificity, it means that the AI software has the potential to serve a s a prescreening tool to alleviate the workload of radiologists in detecting lung cancer, p articularly at a low incidence rate.