Training and Certifying Users of the National Institutes of Health Stroke Scale

Abstract

Background and Purpose- The National Institutes of Health Stroke Scale, designed and validated for use in clinical stroke trials, is now required for all patients with stroke at hospital admission. Recertification is required annually but no data support this frequency; the effect of mandatory training before recertification is unknown. Methods- To clarify optimal recertification frequency and training effect, we assessed users' mastery of the National Institutes of Health Stroke Scale over several years using correct scores (accuracy) on each scale item of the 15-point scale. We also constructed 9 technical errors that could result from misunderstanding the scoring rules. We measured accuracy and the frequency of these technical errors over time. Using multivariable regression, we assessed the effect of time, repeat testing, and profession on user mastery. Results- The final dataset included 1.3×10⁶ examinations. Data were consistent among all 3 online vendors that provide training and certification. Test accuracy showed no significant changes over time. Technical error rates were remarkably low, ranging from 0.48 to 1.36 per 90 test items. Within 2 vendors (that do not require training), the technical error rates increased negligibly over time (P<0.05). In data from a third vendor, mandatory training before recertification improved (reduced) technical errors but not accuracy. Conclusions- The data suggest that mastery of National Institutes of Health Stroke Scale scoring rules is stable over time, and the recertification interval should be lengthened. Mandatory retraining may be needed after unsuccessful recertifications, but not routinely otherwise.

Keywords: certification; clinimetrics; stroke.

Figure One.. Technical Errors for New and Repeat Users in Vendors 1 and 2.

The Technical Error rate in Groups A, B or C is plotted by year as mean and standard deviation. Total number of users in each year is plotted on the right-hand vertical axis.

Figure Two.. Technical Errors for MD and RN users in Vendor 3.

Users’ self-described profession (MD or RN) was used to assess error rates by profession over time. Users repeated certification up to 10 times and the certification groups are labeled on the x-axis. Number of users per data point is labeled. RN users made significantly fewer technical errors than MD users (p<0.05). Over the 10 years, the technical error rates trended higher, reaching statistical significance for Group C (p<0.001).