Examining the Ability of Artificial Neural Networks Machine Learning Models to Accurately Predict Complications Following Posterior Lumbar Spine Fusion

Abstract

Study design: A cross-sectional database study.

Objective: The aim of this study was to train and validate machine learning models to identify risk factors for complications following posterior lumbar spine fusion.

Summary of background data: Machine learning models such as artificial neural networks (ANNs) are valuable tools for analyzing and interpreting large and complex datasets. ANNs have yet to be used for risk factor analysis in orthopedic surgery.

Methods: The American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database was queried for patients who underwent posterior lumbar spine fusion. This query returned 22,629 patients, 70% of whom were used to train our models, and 30% were used to evaluate the models. The predictive variables used included sex, age, ethnicity, diabetes, smoking, steroid use, coagulopathy, functional status, American Society for Anesthesiology (ASA) class ≥3, body mass index (BMI), pulmonary comorbidities, and cardiac comorbidities. The models were used to predict cardiac complications, wound complications, venous thromboembolism (VTE), and mortality. Using ASA class as a benchmark for prediction, area under receiver operating curves (AUC) was used to determine the accuracy of our machine learning models.

Results: On the basis of AUC values, ANN and LR both outperformed ASA class for predicting all four types of complications. ANN was the most accurate for predicting cardiac complications, and LR was most accurate for predicting wound complications, VTE, and mortality, though ANN and LR had comparable AUC values for predicting all types of complications. ANN had greater sensitivity than LR for detecting wound complications and mortality.

Conclusion: Machine learning in the form of logistic regression and ANNs were more accurate than benchmark ASA scores for identifying risk factors of developing complications following posterior lumbar spine fusion, suggesting they are potentially great tools for risk factor analysis in spine surgery.

Level of evidence: 3.

Figure 1.

(A) Schematic of study workflow. (B) Diagram of ANN model. Bar lengths represent number of patient cases. ADASYN increases the number of positive cases to combat class imbalance. Negative cases are then partitioned in a 1:1 ratio with the positive cases to create a class-balanced dataset used for ANN training. Each partition trains an independent neural net. During evaluation, data are fed through each neural net where the responses are surveyed, weighted by the model’s accuracy, and the net prediction is used.

Figure 2.

Coefficient weights obtained from logistic regression analysis used for feature selection. Dark cells indicate highly weighted features indicating a strong predictive value, and lighter cells indicate weakly weighted features.

Figure 3.

Receiver operating curves for ASA, LR, and ANN for each complication type.

Figure 4.

Heatmap of AUC values from LR, ANN, and ASA when predicting cardiac complications (cardiac), VTE, wound complications (wound), and mortality.

Figure 5.

Confusion matrices of trained ANN and LR machine learners evaluated on hold-out (A) mortality and (B) wound complication data sets to demonstrate real-world performance.