Interrater reliability of the Pediatric National Institutes of Health Stroke Scale (PedNIHSS) in a multicenter study

Abstract

Background and purpose: Stroke is an important cause of death and disability among children. Clinical trials for childhood stroke require a valid and reliable acute clinical stroke scale. We evaluated interrater reliability (IRR) of a pediatric adaptation of the National Institutes of Health Stroke Scale.

Methods: The pediatric adaptation of the National Institutes of Health Stroke Scale was developed by pediatric and adult stroke experts by modifying each item of the adult National Institutes of Health Stroke Scale for children, retaining all examination items and scoring ranges of the National Institutes of Health Stroke Scale. Children 2 to 18 years of age with acute arterial ischemic stroke were enrolled in a prospective cohort study from 15 North American sites from January 2007 to October 2009. Examiners were child neurologists certified in the adult National Institutes of Health Stroke Scale. Each subject was examined daily for 7 days or until discharge. A subset of patients at 3 sites was scored simultaneously and independently by 2 study neurologists.

Results: IRR testing was performed in 25 of 113 a median of 3 days (interquartile range, 2 to 4 days) after symptom onset. Patient demographics, total initial pediatric adaptation of the National Institutes of Health Stroke Scale scores, risk factors, and infarct characteristics in the IRR subset were similar to the non-IRR subset. The 2 raters' total scores were identical in 60% and within 1 point in 84%. IRR was excellent as measured by concordance correlation coefficient of 0.97 (95% CI, 0.94 to 0.99); intraclass correlation coefficient of 0.99 (95% CI, 0.97 to 0.99); precision measured by Pearson ρ of 0.97; and accuracy measured by the bias correction factor of 1.0.

Conclusions: There was excellent IRR of the pediatric adaptation of the National Institutes of Health Stroke Scale in a multicenter prospective cohort performed by trained child neurologists.

Figure 1

Scatter plot demonstrating association between total PedNIHSS scores of primary rater (Y-axis) and secondary rater (X-axis). Open symbols depict data for two subjects; closed symbols for single subjects.

Figure 2

Box plot demonstrating total PedNIHSS scores of each rater: Mean (symbol), lower line of box = 25%ile, upper line of box = 75%ile, line in box = median, upper and lower whiskers = largest and smallest non-outlying values, respectively.

Figure 3

Bland and Altman plot of scoring bias. Open symbols depict data for two subjects; closed symbols for single subjects.