Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2016 Jul 19;87(3):249-56.
doi: 10.1212/WNL.0000000000002860. Epub 2016 Jun 17.

Early CT changes in patients admitted for thrombectomy: Intrarater and interrater agreement

Behzad Farzin  1 , Robert Fahed  1 , Francois Guilbert  1 , Alexandre Y Poppe  1 , Nicole Daneault  1 , André P Durocher  1 , Sylvain Lanthier  1 , Hayet Boudjani  1 , Naim N Khoury  1 , Daniel Roy  1 , Alain Weill  1 , Jean-Christophe Gentric  1 , André L Batista  1 , Laurent Létourneau-Guillon  1 , François Bergeron  1 , Marc-Antoine Henry  1 , Tim E Darsaut  1 , Jean Raymond  2
Affiliations
Review

Early CT changes in patients admitted for thrombectomy: Intrarater and interrater agreement

Behzad Farzin et al. Neurology. .

Abstract

Objective: To systematically review the literature and assess agreement on the Alberta Stroke Program Early CT Score (ASPECTS) among clinicians involved in the management of thrombectomy candidates.

Methods: Studies assessing agreement using ASPECTS published from 2000 to 2015 were reviewed. Fifteen raters reviewed and scored the anonymized CT scans of 30 patients recruited in a local thrombectomy trial during 2 independent sessions, in order to study intrarater and interrater agreement. Agreement was measured using intraclass correlation coefficients (ICCs) and Fleiss kappa statistics for ASPECTS and dichotomized ASPECTS at various cutoff values.

Results: The review yielded 30 articles reporting 40 measures of agreement. Populations, methods, analyses, and results were heterogeneous (slight to excellent agreement), precluding a meta-analysis. When analyzed as a categorical variable, intrarater agreement was slight to moderate (κ = 0.042-0.469); it reached a substantial level (κ > 0.6) in 11/15 raters when the score was dichotomized (0-5 vs 6-10). The interrater ICCs varied between 0.672 and 0.811, but agreement was slight to moderate (κ = 0.129-0.315). Even in the best of cases, when ASPECTS was dichotomized as 0-5 vs 6-10, interrater agreement did not reach a substantial level (κ = 0.561), which translates into at least 5 of 15 raters not giving the same dichotomized verdict in 15% of patients.

Conclusions: In patients considered for thrombectomy, there may be insufficient agreement between clinicians for ASPECTS to be reliably used as a criterion for treatment decisions.

PubMed Disclaimer

Figures

Figure 1
Figure 1. Flow diagram of systematic review
ASPECTS = Alberta Stroke Program Early CT Score.
Figure 2
Figure 2. Results of systematic review
Forest plot summarizes the agreement measures retrieved in the systematic review. ASPECTS = Alberta Stroke Program Early CT Score; CI = confidence interval; ICC = intraclass correlation coefficient; IQR = interquartile range; NIHSS = NIH Stroke Scale score.
Figure 3
Figure 3. Graphic representation of the distribution of the Alberta Stroke Program Early CT Score (ASPECTS)
For each patient, represented on the x axis by the mean value of the ASPECTS given by all raters, the distribution of ASPECTS values given by raters is represented on the y-axis by bubbles. The bubble area is proportional to the number of raters who gave the same score (see the bubble scale). Red dots represent the ASPECTS given to each patient by the radiologist on-call. Perfect agreement would have been represented by large bubbles aligned along the diagonal formed by red dots.
Figure 4
Figure 4. Results of interrater agreement study
(A) Graphic display of interrater agreement. (B) Graphic display of agreement with the 16th reader (radiologist on-call), with dichotomized Alberta Stroke Program Early CT Score (ASPECTS) at various cut points. ICC = intraclass correlation coefficient.
Figure 5
Figure 5. Results of intrarater agreement study
Graphic representation of intrarater agreement for each of the 15 readers, expressed as intraclass correlation coefficients (ICCs) or kappa values for the global Alberta Stroke Program Early CT Score (ASPECTS), and kappas obtained for dichotomized scores using various cut points.
See this image and copyright information in PMC

Comment in

References

    1. Barber PA, Demchuk AM, Zhang J, Buchan AM. Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy: ASPECTS Study Group: Alberta Stroke Programme Early CT Score. Lancet 2000;355:1670–1674. - PubMed
    1. Goyal M, Demchuk AM, Menon BK, et al. . Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med 2015;372:1019–1030. - PubMed
    1. Jovin TG, Chamorro A, Cobo E, et al. . Thrombectomy within 8 hours after symptom onset in ischemic stroke. N Engl J Med 2015;372:2296–2306. - PubMed
    1. Saver JL, Goyal M, Bonafe A, et al. . Stent-retriever thrombectomy after intravenous t-Pa vs. t-Pa alone in stroke. N Engl J Med 2015;372:2285–2295. - PubMed
    1. Berkhemer OA, Fransen PS, Beumer D, et al. . A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 2015;372:11–20. - PubMed