Software using artificial intelligence for nodule and cancer detection in CT lung cancer screening: systematic review of test accuracy studies

Abstract

Objectives: To examine the accuracy and impact of artificial intelligence (AI) software assistance in lung cancer screening using CT.

Methods: A systematic review of CE-marked, AI-based software for automated detection and analysis of nodules in CT lung cancer screening was conducted. Multiple databases including Medline, Embase and Cochrane CENTRAL were searched from 2012 to March 2023. Primary research reporting test accuracy or impact on reading time or clinical management was included. QUADAS-2 and QUADAS-C were used to assess risk of bias. We undertook narrative synthesis.

Results: Eleven studies evaluating six different AI-based software and reporting on 19 770 patients were eligible. All were at high risk of bias with multiple applicability concerns. Compared with unaided reading, AI-assisted reading was faster and generally improved sensitivity (+5% to +20% for detecting/categorising actionable nodules; +3% to +15% for detecting/categorising malignant nodules), with lower specificity (-7% to -3% for correctly detecting/categorising people without actionable nodules; -8% to -6% for correctly detecting/categorising people without malignant nodules). AI assistance tended to increase the proportion of nodules allocated to higher risk categories. Assuming 0.5% cancer prevalence, these results would translate into additional 150-750 cancers detected per million people attending screening but lead to an additional 59 700 to 79 600 people attending screening without cancer receiving unnecessary CT surveillance.

Conclusions: AI assistance in lung cancer screening may improve sensitivity but increases the number of false-positive results and unnecessary surveillance. Future research needs to increase the specificity of AI-assisted reading and minimise risk of bias and applicability concerns through improved study design.

Prospero registration number: CRD42021298449.

Keywords: Clinical Epidemiology; Imaging/CT MRI etc; Lung Cancer; Non-Small Cell Lung Cancer.

Figure 1. Accuracy of readers (nodule detection; nodule categorisation based on volume measurement; or nodule detection plus risk categorisation and recall decision for lung cancer diagnosis) both with and without concurrent AI use (seven studies with comparative data). Estimates connected with a line are from the same study. ¹ Zhang et al; ² Hsu et al; ³ Lo et al; ⁴ Singh et al; ⁵ Lancaster et al; ⁶ Hwang et al; ⁷ Park et al. *Data from Hall et al are not presented as the study compared AI-assisted reading by radiographers against unaided radiologists, which differed in nature from the other studies. AI, artificial intelligence; Lung-RADS, Lung CT Screening Reporting & Data System.