Machine-learning Models Predict 30-Day Mortality, Cardiovascular Complications, and Respiratory Complications After Aseptic Revision Total Joint Arthroplasty

Abstract

Background: Aseptic revision THA and TKA are associated with an increased risk of adverse outcomes compared with primary THA and TKA. Understanding the risk profiles for patients undergoing aseptic revision THA or TKA may provide an opportunity to decrease the risk of postsurgical complications. There are risk stratification tools for postoperative complications after aseptic revision TKA or THA; however, current tools only include nonmodifiable risk factors, such as medical comorbidities, and do not include modifiable risk factors.

Questions/purposes: (1) Can machine learning predict 30-day mortality and complications for patients undergoing aseptic revision THA or TKA using a cohort from the American College of Surgeons National Surgical Quality Improvement Program database? (2) Which patient variables are the most relevant in predicting complications?

Methods: This was a temporally validated, retrospective study analyzing the 2014 to 2019 National Surgical Quality Improvement Program database, as this database captures a large cohort of aseptic revision THA and TKA patients across a broad range of clinical settings and includes preoperative laboratory values. The training data set was 2014 to 2018, and 2019 was the validation data set. Given that predictive models learn expected prevalence of outcomes, this split allows assessment of model performance in contemporary patients. Between 2014 and 2019, a total of 24,682 patients underwent aseptic revision TKA and 17,871 patients underwent aseptic revision THA. Of those, patients with CPT codes corresponding to aseptic revision TKA or THA were considered as potentially eligible. Based on excluding procedures involving unclean wounds, 78% (19,345 of 24,682) of aseptic revision TKA procedures and 82% (14,711 of 17,871) of aseptic revision THA procedures were eligible. Ten percent of patients in each of the training and validation cohorts had missing predictor variables. Most of these missing data were preoperative sodium or hematocrit (8% in both the training and validation cohorts). No patients had missing outcome data. No patients were excluded due to missing data. The mean patient was age 66 ± 12 years, the mean BMI was 32 ± 7 kg/m 2 , and the mean American Society of Anesthesiologists (ASA) Physical Score was 3 (56%). XGBoost was then used to create a scoring tool for 30-day adverse outcomes. XGBoost was chosen because it can handle missing data, it is nonlinear, it can assess nuanced relationships between variables, it incorporates techniques to reduce model complexity, and it has a demonstrated record of producing highly accurate machine-learning models. Performance metrics included discrimination and calibration. Discrimination was assessed by c-statistics, which describe the area under the receiver operating characteristic curve. This quantifies how well a predictive model discriminates between patients who have the outcome of interest versus those who do not. Relevant ranges for c-statistics include good (0.70 to 0.79), excellent (0.80 to 0.89), and outstanding (> 0.90). We estimated 95% confidence intervals (CIs) for c-statistics by 500-sample bootstrapping. Calibration curves quantify reliability of model predictions. Reliable models produce prediction probabilities for outcomes that are similar to observed probabilities of those outcomes, so a well-calibrated model should demonstrate a calibration curve that does not deviate substantially from a line of slope 1 and intercept 0. Calibration curves were generated on the 2019 validation data. Shapley Additive Explanations (SHAP) visualizations were used to investigate feature importance to gain insight into how models made predictions. The models were built into an online calculator for ongoing testing and validation. The risk calculator, which is freely available ( http://nb-group.org/rev2/ ), allows a user to input patient data to calculate postoperative risk of 30-day mortality, cardiac, and respiratory complications after aseptic revision TKA or THA. A post hoc analysis was performed to assess whether using data from 2020 would improve calibration on 2019 data.

Results: The model accurately predicted mortality, cardiac complications, and respiratory complications after aseptic revision THA or TKA, with c-statistics of 0.88 (95% CI 0.83 to 0.93), 0.80 (95% CI 0.75 to 0.84), and 0.78 (95% CI 0.74 to 0.82), respectively, on internal validation and 0.87 (95% CI 0.77 to 0.96), 0.70 (95% CI 0.61 to 0.78), and 0.82 (95% CI 0.75 to 0.88), respectively, on temporal validation. Calibration curves demonstrated slight over-confidence in predictions (most predicted probabilities were higher than observed probabilities). Post hoc analysis of 2020 data did not yield improved calibration on the 2019 validation set. Important risk factors for all models included increased age and higher ASA, BMI, hematocrit level, and sodium level. Hematocrit and ASA were in the top three most important features for all models. The factor with the strongest association for mortality and cardiac complication models was age, and for the respiratory model, chronic obstructive pulmonary disease. Risk related to sodium followed a U-shaped curve. Preoperative hyponatremia and hypernatremia predicted an increased risk of mortality and respiratory complications, with a nadir of 138 mmol/L; hyponatremia was more strongly associated with mortality than hypernatremia. A hematocrit level less than 36% predicted an increased risk of all three adverse outcomes. A BMI less than 24 kg/m 2 -and especially less than 20 kg/m 2 -predicted an increased risk of all three adverse outcomes, with little to no effect for higher BMI.

Conclusion: This temporally validated model predicted 30-day mortality, cardiac complications, and respiratory complications after aseptic revision THA or TKA with c-statistics ranging from 0.78 to 0.88. This freely available risk calculator can be used preoperatively by surgeons to educate patients on their individual postoperative risk of these specific adverse outcomes. Unanswered questions that remain include whether altering the studied preoperative patient variables, such as sodium or hematocrit, would affect postoperative risk of adverse outcomes; however, a prospective cohort study is needed to answer this question.

Level of evidence: Level III, therapeutic study.

Fig. 1.

This graph shows a SHAP feature importance plot for the model predicting mortality after aseptic revision TKA or THA. SHAP values quantify the effect that each feature has on model predictions. Features are ordered from top to bottom according to the magnitude each contributed to making model predictions. Colors correlate with feature value. For example, blue corresponds to younger age and lower sodium values, red corresponds to older age and higher sodium values. We have also gathered granular details on exact feature values (Appendix 1).

Fig. 2.

This calibration plot represents reliability of model predictions on mortality after aseptic revision TKA or THA. By plotting the observed probability of outcomes that were actually seen in the data versus predicted probability of outcomes, this calibration plot demonstrates whether model predictions for mortality tended toward over- or underpredicting risk. Predictions using the model trained on 2014 to 2018 data are consistently higher than observations from 2019.

Fig. 3.

This calibration plot represents the reliability of model predictions on cardiac complications after aseptic revision TKA or THA. By plotting the observed probability of outcomes that were actually seen in the data versus predicted probability of outcomes, this calibration plot demonstrates whether model predictions for cardiac complications tended toward over- or underpredicting risk. Above approximately 0.3%, risk predictions using the model trained on 2014 to 2018 data are consistently slightly higher than observations from 2019.

Fig. 4.

This calibration plot represents the reliability of model predictions on respiratory complications after aseptic revision TKA or THA. By plotting observed probability of outcomes that were actually seen in the data versus predicted probability of outcomes, this calibration plot demonstrates whether model predictions for respiratory complications tended toward over- or underpredicting risk. Predictions using the model trained on 2014 to 2018 data are higher than observed probabilities below 1.25% and lower than observed probabilities above 1.25%. This overestimation in risk below 1.25% is driven by higher incidence of respiratory complications in the 2014 to 2018 data compared with 2019.