Validity of stroke severity assessment using medical records in a population-based cohort

Abstract

Objectives: Stroke severity is an important prognostic indicator of morbidity and mortality, but often not recorded outside of specialised stroke centres. We aimed to develop a scoring rule and validate standardised assessment of the National Institutes of Health Stroke Scale (NIHSS) from medical records.

Methods: We developed a standardised assessment of the NIHSS from medical records. Four trained raters independently assessed the charts of 100 patients with first-ever stroke, randomly selected from the population-based Rotterdam Study cohort. Interrater agreement was determined using the intraclass correlation coefficient (ICC), and Fleiss' kappa for major versus minor stroke. We validated the scoring method against 29 prospective, clinical NIHSS ratings, using Kendall's tau and Cohen's kappa.

Results: Of 100 included patients with stroke (mean age 80 years, 62% women), 71 (71%) were admitted to hospital and 9 (9%) were seen in outpatient clinic, whereas 20 (20%) were treated exclusively by their general practitioner or nursing home physician. Interrater agreement for retrospective, chart-based NIHSS ratings was excellent when assessed continuously (ICC: 0.90), and for minor versus major stroke (for NIHSS>3: κ=0.79, NIHSS>5: κ=0.78). Interrater agreement was good both for hospital-based and out-of-hospital settings (ICC: 0.97 and 0.75 respectively). Overall, assessment from medical records was in excellent agreement with prospective NIHSS ratings (τ=0.83; NIHSS>3: κ=0.93, and NIHSS>5: κ=0.93). However, for severe stroke (NIHSS>10) retrospective assessment tended to underestimate severity by 1-3 points on the NIHSS, which was accompanied by a somewhat lower interrater agreement for those more severe cases (NIHSS>10: κ=0.62).

Conclusions: Assessment of stroke severity according to the NIHSS on the basis of medical records is feasible and reliable in population-based cohorts of patients with stroke. These findings facilitate more individualised risk estimates in observational studies that lack prospective ascertainment of stroke severity.

Keywords: National Institutes of Health Stroke Scale; Population-based; Stroke; Stroke severity.