Automatic Grading of Stroke Symptoms for Rapid Assessment Using Optimized Machine Learning and 4-Limb Kinematics: Clinical Validation Study

Abstract

Background: Subtle abnormal motor signs are indications of serious neurological diseases. Although neurological deficits require fast initiation of treatment in a restricted time, it is difficult for nonspecialists to detect and objectively assess the symptoms. In the clinical environment, diagnoses and decisions are based on clinical grading methods, including the National Institutes of Health Stroke Scale (NIHSS) score or the Medical Research Council (MRC) score, which have been used to measure motor weakness. Objective grading in various environments is necessitated for consistent agreement among patients, caregivers, paramedics, and medical staff to facilitate rapid diagnoses and dispatches to appropriate medical centers.

Objective: In this study, we aimed to develop an autonomous grading system for stroke patients. We investigated the feasibility of our new system to assess motor weakness and grade NIHSS and MRC scores of 4 limbs, similar to the clinical examinations performed by medical staff.

Methods: We implemented an automatic grading system composed of a measuring unit with wearable sensors and a grading unit with optimized machine learning. Inertial sensors were attached to measure subtle weaknesses caused by paralysis of upper and lower limbs. We collected 60 instances of data with kinematic features of motor disorders from neurological examination and demographic information of stroke patients with NIHSS 0 or 1 and MRC 7, 8, or 9 grades in a stroke unit. Training data with 240 instances were generated using a synthetic minority oversampling technique to complement the imbalanced number of data between classes and low number of training data. We trained 2 representative machine learning algorithms, an ensemble and a support vector machine (SVM), to implement auto-NIHSS and auto-MRC grading. The optimized algorithms performed a 5-fold cross-validation and were searched by Bayes optimization in 30 trials. The trained model was tested with the 60 original hold-out instances for performance evaluation in accuracy, sensitivity, specificity, and area under the receiver operating characteristics curve (AUC).

Results: The proposed system can grade NIHSS scores with an accuracy of 83.3% and an AUC of 0.912 using an optimized ensemble algorithm, and it can grade with an accuracy of 80.0% and an AUC of 0.860 using an optimized SVM algorithm. The auto-MRC grading achieved an accuracy of 76.7% and a mean AUC of 0.870 in SVM classification and an accuracy of 78.3% and a mean AUC of 0.877 in ensemble classification.

Conclusions: The automatic grading system quantifies proximal weakness in real time and assesses symptoms through automatic grading. The pilot outcomes demonstrated the feasibility of remote monitoring of motor weakness caused by stroke. The system can facilitate consistent grading with instant assessment and expedite dispatches to appropriate hospitals and treatment initiation by sharing auto-MRC and auto-NIHSS scores between prehospital and hospital responses as an objective observation.

Keywords: artificial intelligence; kinematics; machine learning; sensors; stroke; telemedicine.

Figure 1

Patient enrollment and data set for automatic grading system. MRC: Medical Research Council; NIHSS: National Institutes of Health Stroke Scale; SMOTE: synthetic minority oversampling technique.

Figure 2

Automatic grading process. MRC: Medical Research Council; NIHSS: National Institutes of Health Stroke Scale.

Figure 3

Schematic of upper and lower limb sensors and corresponding segment axes.

Figure 4

Grade distribution of NIHSS and MRC. MRC: Medical Research Council; NIHSS: National Institutes of Health Stroke Scale.

Figure 5

Statistical plots of 4-limb features of NIHSS grades. NIHSS: National Institutes of Health Stroke Scale.

Figure 6

Confusion matrix and receiver operating characteristic of auto-NIHSS grading using (A) support vector machine and (B) ensemble learning. AUC: area under the receiver operating characteristics curve; NIHSS: National Institutes of Health Stroke Scale.

Figure 7

Statistical plots of 4-limb features of MRC grades. MRC: Medical Research Council.

Figure 8

Confusion matrix and receiver operating characteristic of auto-MRC grading using (A) support vector machine and (B) ensemble learning. AUC: area under the receiver operating characteristics curve; MRC: Medical Research Council.