Distributional Validity and Prognostic Power of the National Institutes of Health Stroke Scale in US Administrative Claims Data

Abstract

Importance: Comparative assessment of acute ischemic stroke care quality provided by hospitals in the United States has been hampered by the unavailability of the National Institutes of Health Stroke Scale (NIHSS) in administrative data sets, preventing adequate adjustment for variations in patient case-mix risk. In response to stakeholder concerns, the US Centers for Medicare & Medicaid Services in 2016 implemented optional reporting of NIHSS scores.

Objective: To analyze the distributional, convergent, and predictive validity of nationally submitted NIHSS values in the National Inpatient Sample.

Design, setting, and participants: This population-based retrospective cross-sectional study took place from October 1 to December 31, 2016. The nationally representative sample included US adults who had ischemic stroke hospitalizations during the first calendar quarter in which optional NIHSS reporting was implemented. Analysis began September 2019.

Main outcomes and measures: Distribution of NIHSS scores, functional independence at discharge, inpatient mortality, and administrative reporting of NIHSS.

Results: Among 154 165 ischemic stroke hospitalizations during the first 3 months of the reporting policy, NIHSS scores were reported in 21 685 patients (14%) (10 925 women [50.4%]; median [interquartile range] age, 72 [61-82] years). Median (interquartile range) NIHSS score was 4 (2-11), and frequency of severity categories included absent (NIHSS score, 0) in 2080 patients (9.6%), minor (NIHSS score, 1-4) in 8760 patients (40.4%), and severe (NIHSS score, 21-42) in 1930 patients (8.9%). National Institutes of Health Stroke Scale score of 10 or more, an indicator of possible large vessel occlusions, was present in 6290 patients (29%). Presenting NIHSS score was higher in very elderly patients (age ≥80 y) and women and also in patients receiving endovascular thrombectomy vs intravenous thrombolysis alone vs no reperfusion therapy (median [interquartile range], 17 [12-22] vs 6 [4-12] vs 4 [2-9], respectively) (P < .001). National Institutes of Health Stroke Scale scores were similarly higher for discharge outcomes of mortality vs discharge to skilled nursing facility vs discharge home (median [interquartile range], 19 [12-25] vs 7 [3-15] vs 2 [1-5], respectively) (P < .001). Likelihood of NIHSS scores being reported independently increased with interfacility transfer, receipt of acute reperfusion therapies, larger hospital size, academic centers, and region other than the West.

Conclusions and relevance: In the initial national optional reporting period in the United States, NIHSS scores were reported in nearly 1 in 7 ischemic stroke hospitalizations. The distribution of NIHSS scores was similar to that from narrow population-based studies and registries, and NIHSS scores were powerfully associated with discharge outcome, supporting the validity and potential to aid care quality assessment.

Figure 1.. Distribution of Administratively Recorded National Institutes of Health Stroke Scale (NIHSS) Scores in Ischemic Stroke Hospitalizations in US National Inpatient Sample

Figure 2.. Distributions of Administratively Recorded NIHSS Scores in Patients With Excellent (Discharged Home) and Fatal Outcomes at Completion of Acute Stroke Hospitalization in US National Inpatient Sample

^aNo patient had a National Institutes of Health Stroke Scale (NIHSS) score of 40.

Figure 3.. Distribution of Median NIHSS Scores Among Quintiles of Hospital NIHSS Reporting Performance

From lowest to highest quintile of reporting frequency, median National Institutes of Health Stroke Scale (NIHSS) values were 5, 4.5, 4.5, 4.25, and 3, respectively (P = .06).