Classifying High-Risk Patients for Persistent Opioid Use After Major Spine Surgery: A Machine-Learning Approach

Abstract

Background: Persistent opioid use is a common occurrence after surgery and prolonged exposure to opioids may result in escalation and dependence. The objective of this study was to develop machine-learning-based predictive models for persistent opioid use after major spine surgery.

Methods: Five classification models were evaluated to predict persistent opioid use: logistic regression, random forest, neural network, balanced random forest, and balanced bagging. Synthetic Minority Oversampling Technique was used to improve class balance. The primary outcome was persistent opioid use, defined as patient reporting to use opioids after 3 months postoperatively. The data were split into a training and test set. Performance metrics were evaluated on the test set and included the F1 score and the area under the receiver operating characteristics curve (AUC). Feature importance was ranked based on SHapley Additive exPlanations (SHAP).

Results: After exclusion (patients with missing follow-up data), 2611 patients were included in the analysis, of which 1209 (46.3%) continued to use opioids 3 months after surgery. The balanced random forest classifiers had the highest AUC (0.877, 95% confidence interval [CI], 0.834-0.894) compared to neural networks (0.729, 95% CI, 0.672-0.787), logistic regression (0.709, 95% CI, 0.652-0.767), balanced bagging classifier (0.859, 95% CI, 0.814-0.905), and random forest classifier (0.855, 95% CI, 0.813-0.897). The balanced random forest classifier had the highest F1 (0.758, 95% CI, 0.677-0.839). Furthermore, the specificity, sensitivity, precision, and accuracy were 0.883, 0.700, 0.836, and 0.780, respectively. The features based on SHAP analysis with the highest impact on model performance were age, preoperative opioid use, preoperative pain scores, and body mass index.

Conclusions: The balanced random forest classifier was found to be the most effective model for identifying persistent opioid use after spine surgery.

Figure 1.

Analysis pipeline workflow. AUC indicates area under the receiver operating characteristic curve; EHR, electronic health record; LOS, length of stay; SHAP, SHapley Additive exPlanations; SMOTE, Synthetic Minority Oversampling Technique.

Figure 2.

Performance––based on AUC––of each machine-learning model when tested on the test set (20% of dataset held out of training) when (A) not using SMOTE or (B) using SMOTE. AUC indicates area under the receiver operating characteristics curve; SMOTE, Synthetic Minority Oversampling Technique.

Figure 3.

A calibration plot visualizing the fit of the best model on the test set. The predicted risk was plotted against the observed risk for each of the risk percentiles created from the data set.

Figure 4.

Beeswarm plot illustrating SHAP values for feature importance based on the random forest classifier. NRS indicates numeric rating scale; SHAP, SHapley Additive exPlanations.