Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Aug 10;9(8):e19065.
doi: 10.1016/j.heliyon.2023.e19065. eCollection 2023 Aug.

AI-support for the detection of intracranial large vessel occlusions: One-year prospective evaluation

K G van Leeuwen  1 , M J Becks  1 , D Grob  1 , F de Lange  1 , J H E Rutten  1 , S Schalekamp  1 , M J C M Rutten  1   2 , B van Ginneken  1 , M de Rooij  1 , F J A Meijer  1
Affiliations

AI-support for the detection of intracranial large vessel occlusions: One-year prospective evaluation

K G van Leeuwen et al. Heliyon. .

Abstract

Purpose: Few studies have evaluated real-world performance of radiological AI-tools in clinical practice. Over one-year, we prospectively evaluated the use of AI software to support the detection of intracranial large vessel occlusions (LVO) on CT angiography (CTA).

Method: Quantitative measures (user log-in attempts, AI standalone performance) and qualitative data (user surveys) were reviewed by a key-user group at three timepoints. A total of 491 CTA studies of 460 patients were included for analysis.

Results: The overall accuracy of the AI-tool for LVO detection and localization was 87.6%, sensitivity 69.1% and specificity 91.2%. Out of 81 LVOs, 31 of 34 (91%) M1 occlusions were detected correctly, 19 of 38 (50%) M2 occlusions, and 6 of 9 (67%) ICA occlusions. The product was considered user-friendly. The diagnostic confidence of the users for LVO detection remained the same over the year. The last measured net promotor score was -56%. The use of the AI-tool fluctuated over the year with a declining trend.

Conclusions: Our pragmatic approach of evaluating the AI-tool used in clinical practice, helped us to monitor the usage, to estimate the perceived added value by the users of the AI-tool, and to make an informed decision about the continuation of the use of the AI-tool.

Keywords: Artificial intelligence; Cerebrovascular occlusion; Computed tomography angiography; Evaluation study; Stroke.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Figures

Fig. 1
Fig. 1
Flowchart of data included in the evaluation.
Fig. 2
Fig. 2
AI-tool use (radiologists and residents) per month. Figure a) shows the login attempts and b) the unique users. Multiple login attempts within 1 h by a single user, where considered as a single login.
See this image and copyright information in PMC

References

    1. Goyal M., Menon B.K., Zwam van W.H., et al. Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet. 2016;387:1723–1731. doi: 10.1016/S0140-6736(16)00163-X. - DOI - PubMed
    1. Saver J.L., Goyal M., Bonafe A., et al. Stent-retriever thrombectomy after intravenous t-PA vs. t-PA alone in stroke. N. Engl. J. Med. 2015;372:2285–2295. doi: 10.1056/NEJMoa1415061. - DOI - PubMed
    1. Saver J.L., Goyal M., Lugt van der A., et al. Time to treatment with endovascular thrombectomy and outcomes from ischemic stroke: a meta-analysis. JAMA. 2016;316:1279–1288. doi: 10.1001/jama.2016.13647. - DOI - PubMed
    1. Diagnostic Imaging Analysis Group . Radboud university medical center; 2020. AI for Radiology.https://www.aiforradiology.com
    1. Data Science Institute . American College of Radiology; 2021. AI Central.https://aicentral.acrdsi.org/

LinkOut - more resources