Non-radiology Healthcare Professionals Significantly Benefit from AI-Assistance in Emergency-Related Chest Radiography Interpretation

Science and Research

BACKGROUND: Chest radiography (CXR) is still of crucial importance in primary diagnostics, but interpretation poses difficulties at times. RESEARCH QUESTION: Can a convolutional neural network (CNN)-based AI system that interprets CXRs add value in an emergency unit (EU) setting? STUDY DESIGN AND METHODS: A total of 563 CXRs acquired in the EU of a major university hospital were retrospectively assessed twice by three board-certified radiologists (BCRs), three radiology residents (RRs), and three EU-experienced non-radiology residents (NRRs) employing a two-step reading process: (1) without AI support (woAI), (2) with AI support providing additional images with AI overlays (wAI). Suspicion of four suspected pathologies (pleural effusion, pneumothorax, consolidations suspicious for pneumonia, nodules) was reported on a 5-point confidence scale. BCRs' confidence scores were converted into four binary reference standards (RFS I-IV) of different sensitivities. RRs' and NRRs' performances woAI/wAI were statistically compared using receiver operating characteristics (ROCs), Youden statistics and operating point metrics derived from fitted ROC curves. RESULTS: NRRs could significantly improve performance, sensitivity and accuracy wAI in all four pathologies tested. E.g., in the most sensitive RFS IV, NRR consensus improved the AUC (mean, 95%-confidence-interval) in the detection of the time-critical pathology pneumothorax from 0.846 (0.785-0.907) woAI to 0.974 (0.947-1.000) wAI [p<0.001], which represented a gain of 30% in sensitivity and 2% in accuracy (while maintaining an optimized specificity). The most pronounced effect was observed in nodule detection, with NRR wAI improving sensitivity by 53% and accuracy by 7% (AUC woAI: 0.723 [0.661-0.785], wAI: 0.890 [0.848-0.931], p<0.001). The RR consensus wAI showed smaller, mostly non-significant gains in performance, sensitivity and accuracy. INTERPRETATION: In an EU setting without 24/7 radiology coverage, the presented AI solution features an excellent clinical support tool to non-radiologists, similar to a second reader, and allows for a more accurate primary diagnosis and thus earlier therapy initiation.