Accuracy of artificial intelligence software for CT angiography in stroke

Abstract

Objective: Software developed using artificial intelligence may automatically identify arterial occlusion and provide collateral vessel scoring on CT angiography (CTA) performed acutely for ischemic stroke. We aimed to assess the diagnostic accuracy of e-CTA by Brainomix™ Ltd by large-scale independent testing using expert reading as the reference standard.

Methods: We identified a large clinically representative sample of baseline CTA from 6 studies that recruited patients with acute stroke symptoms involving any arterial territory. We compared e-CTA results with masked expert interpretation of the same scans for the presence and location of laterality-matched arterial occlusion and/or abnormal collateral score combined into a single measure of arterial abnormality. We tested the diagnostic accuracy of e-CTA for identifying any arterial abnormality (and in a sensitivity analysis compliant with the manufacturer's guidance that software only be used to assess the anterior circulation).

Results: We include CTA from 668 patients (50% female; median: age 71 years, NIHSS 9, 2.3 h from stroke onset). Experts identified arterial occlusion in 365 patients (55%); most (343, 94%) involved the anterior circulation. Software successfully processed 545/668 (82%) CTAs. The sensitivity, specificity and diagnostic accuracy of e-CTA for detecting arterial abnormality were each 72% (95% CI = 66-77%). Diagnostic accuracy was non-significantly improved in a sensitivity analysis excluding occlusions from outside the anterior circulation (76%, 95% CI = 72-80%).

Interpretation: Compared to experts, the diagnostic accuracy of e-CTA for identifying acute arterial abnormality was 72-76%. Users of e-CTA should be competent in CTA interpretation to ensure all potential thrombectomy candidates are identified.

Figure 1

Flowchart of individual study patient contribution and CTA image processing in RITeS.

Figure 2

Potential real‐world implications of diagnostic accuracy results for e‐CTA identification of acute angiographic abnormality (arterial occlusion and/or abnormal collateral scoring) compared to masked expert reference standard. (A) Patients with angiographic abnormality. True positive = white. False negative = grey. (B) Patients without angiographic abnormality. True negative = horizontal lines. False positive = black. These analyses include 4% of cases with imaging features outside the intended use of software: non‐ICA/MCA occlusion.