Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Dec;31(4):1093-1100.
doi: 10.1007/s00062-020-00988-x. Epub 2021 Jan 27.

ASPECTS Interobserver Agreement of 100 Investigators from the TENSION Study

Noel van Horn  1 , Helge Kniep  2 , Gabriel Broocks  2 , Lukas Meyer  2 , Fabian Flottmann  2 , Matthias Bechstein  2 , Julia Götz  2 , Götz Thomalla  3 , Martin Bendszus  4 , Susanne Bonekamp  4 , Johannes Alex Rolf Pfaff  4 , Paulo Roberto Dellani  5 , Jens Fiehler  2 , Uta Hanning  2
Affiliations

ASPECTS Interobserver Agreement of 100 Investigators from the TENSION Study

Noel van Horn et al. Clin Neuroradiol. 2021 Dec.

Abstract

Purpose: Evaluating the extent of cerebral ischemic infarction is essential for treatment decisions and assessment of possible complications in patients with acute ischemic stroke. Patients are often triaged according to image-based early signs of infarction, defined by Alberta Stroke Program Early CT Score (ASPECTS). Our aim was to evaluate interrater reliability in a large group of readers.

Methods: We retrospectively analyzed 100 investigators who independently evaluated 20 non-contrast computed tomography (NCCT) scans as part of their qualification program for the TENSION study. Test cases were chosen by four neuroradiologists who had previously scored NCCT scans with ASPECTS between 0 and 8 and high interrater agreement. Percent and interrater agreements were calculated for total ASPECTS, as well as for each ASPECTS region.

Results: Percent agreements for ASPECTS ratings was 28%, with interrater agreement of 0.13 (95% confidence interval, CI 0.09-0.16), at zero tolerance allowance and 66%, with interrater agreement of 0.32 (95% CI: 0.21-0.44), at tolerance allowance set by TENSION inclusion criteria. ASPECTS region with highest level of agreement was the insular cortex (percent agreement = 96%, interrater agreement = 0.96 (95% CI: 0.94-0.97)) and with lowest level of agreement the M3 region (percent agreement = 68%, interrater agreement = 0.39 [95% CI: 0.17-0.61]).

Conclusion: Interrater agreement reliability for total ASPECTS and study enrollment was relatively low but seems sufficient for practical application. Individual region analysis suggests that some are particularly difficult to evaluate, with varying levels of reliability. Potential impairment of the supraganglionic region must be examined carefully, particularly with respect to the decision whether or not to perform mechanical thrombectomy.

Keywords: Acute stroke therapy; Brain; Endovascular treatment; Interrater reliability; Ischemic stroke; Krippendorff’s α.

PubMed Disclaimer

Conflict of interest statement

G. Thomalla: consulting fees from Acandis, grant support and lecture fees from Bayer, lecture fees from Boehringer Ingelheim, Bristol-Myers Squibb/Pfizer, and Daiichi Sankyo, and consulting fees and lecture fees from Stryker. Grants from Bundesministerium für Wirtschaft und Energie (BMWi), Deutsche Forschungsgemeinschaft (DFG), European Union (EU), German Innovation Fund, Corona Foundation. J.A.R. Pfaff: personal fees from Stryker, outside the submitted work. J. Fiehler: consultant for Acandis, Boehringer Ingelheim, Codman, Microvention, Sequent, Stryker. Speaker for Bayer Healthcare, Bracco, Covidien/ev3, Penumbra, Philips, Siemens. Grants from Bundesministeriums für Wirtschaft und Energie (BMWi), Bundesministerium für Bildung und Forschung (BMBF), Deutsche Forschungsgemeinschaft (DFG), European Union (EU), Covidien, Stryker (THRILL study), Microvention (ERASER study), Philips. N. van Horn, H. Kniep, G. Broocks, L. Meyer, F. Flottmann, M. Bechstein, J. Götz, M. Bendszus, S. Bonekamp, P.R. Dellani, and U. Hanning declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Frequency distribution of Alberta Stroke Program Early CT Score (ASPECTS) ratings for each of the 20 cases
Fig. 2
Fig. 2
Illustrates ASPECTS per region agreement on the level of the basal ganglia (a) and immediately above (b) calculated using Gwet’s AC1 on NCCT with low (red), moderate (yellow) and high (green) agreement. I insula, IC internal capsule, C caput, L lentiform, M1–M6 media cerebral artery territory M1–M6, ASPECTS Alberta Stroke Program Early CT Score, NCCT non-contrast computed tomography
Fig. 3
Fig. 3
Demonstrates rating concordance among all raters regarding ASPECTS rating of all 20 cases for each ASPECTS region, mean concordance for all cases and concordance for enrolment decision for mechanical thrombectomy. Mean ASPECTS is illustrated according to ASPECTS with standard deviation (standard dev) sorted by lowest to highest standard dev (blue, high concordance; red, low concordance). I insula, IC internal capsule, C caput, L lentiform, M1–M6 media cerebral artery territory M1–M6, ASPECTS Alberta Stroke Program Early CT Score
See this image and copyright information in PMC

References

    1. Barber PA, Demchuk AM, Zhang J, Buchan AM. Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy. ASPECTS Study Group. Alberta Stroke Programme Early CT Score. Lancet. 2000;355:1670–1674. doi: 10.1016/S0140-6736(00)02237-6. - DOI - PubMed
    1. Yoo AJ, Zaidat OO, Chaudhry ZA, Berkhemer OA, González RG, Goyal M, Demchuk AM, Menon BK, Mualem E, Ueda D, Buell H, Sit SP, Bose A, Penumbra Pivotal and Penumbra Imaging Collaborative Study (PICS) Investigators Impact of pretreatment noncontrast CT Alberta Stroke Program Early CT Score on clinical outcome after intra-arterial stroke therapy. Stroke. 2014;45:746–751. doi: 10.1161/STROKEAHA.113.004260. - DOI - PubMed
    1. Goyal M, Menon BK, Coutts SB, Hill MD, Demchuk AM, Penumbra Pivotal Stroke Trial Investigators and the Seaman MR Research Center Effect of baseline CT scan appearance and time to recanalization on clinical outcomes in endovascular thrombectomy of acute ischemic strokes. Stroke. 2011;42:93–97. doi: 10.1161/STROKEAHA.110.594481. - DOI - PubMed
    1. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, Biller J, Brown M, Demaerschalk BM, Hoh B, Jauch EC, Kidwell CS, Leslie-Mazwi TM, Ovbiagele B, Scott PA, Sheth KN, Southerland AM, Summers DV, Tirschwell DL. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke. 2019;50:e344–418. doi: 10.1161/STROKEAHA.118.022606. - DOI - PubMed
    1. Turc G, Bhogal P, Fischer U, Khatri P, Lobotesis K, Mazighi M, Schellinger PD, Toni D, de Vries J, White P. European Stroke Organisation (ESO)—European Society for Minimally Invasive Neurological Therapy (ESMINT) Guidelines on Mechanical Thrombectomy in Acute Ischemic Stroke. J Neurointerv Surg. 2019 doi: 10.1136/neurintsurg-2018-014569. - DOI - PubMed