Predicting 10-Year Risk of End-Organ Complications of Type 2 Diabetes With and Without Metabolic Surgery: A Machine Learning Approach

Abstract

Objective: To construct and internally validate prediction models to estimate the risk of long-term end-organ complications and mortality in patients with type 2 diabetes and obesity that can be used to inform treatment decisions for patients and practitioners who are considering metabolic surgery.

Research design and methods: A total of 2,287 patients with type 2 diabetes who underwent metabolic surgery between 1998 and 2017 in the Cleveland Clinic Health System were propensity-matched 1:5 to 11,435 nonsurgical patients with BMI ≥30 kg/m² and type 2 diabetes who received usual care with follow-up through December 2018. Multivariable time-to-event regression and random forest machine learning models were built and internally validated using fivefold cross-validation to predict the 10-year risk for four outcomes of interest. The prediction models were programmed to construct user-friendly web-based and smartphone applications of Individualized Diabetes Complications (IDC) Risk Scores for clinical use.

Results: The prediction tools demonstrated the following discrimination ability based on the area under the receiver operating characteristic curve (1 = perfect discrimination and 0.5 = chance) at 10 years in the surgical and nonsurgical groups, respectively: all-cause mortality (0.79 and 0.81), coronary artery events (0.66 and 0.67), heart failure (0.73 and 0.75), and nephropathy (0.73 and 0.76). When a patient's data are entered into the IDC application, it estimates the individualized 10-year morbidity and mortality risks with and without undergoing metabolic surgery.

Conclusions: The IDC Risk Scores can provide personalized evidence-based risk information for patients with type 2 diabetes and obesity about future cardiovascular outcomes and mortality with and without metabolic surgery based on their current status of obesity, diabetes, and related cardiometabolic conditions.

Figure 1

Calibration of cross-validated 10-year risks for each outcome, stratified by treatment group. Cross-validated risks were binned into 10 subgroups, and each subgroup’s average risk was plotted against the observed cumulative incidence of the subgroup (Kaplan-Meier for all-cause mortality). Local regression smoothers were then drawn through the points. The closer the points lie along the 45° line, the better the calibration.

Figure 2

The relative rankings of importance of each baseline variable in final prediction models for each outcome (all-cause mortality [A], coronary artery disease [B], heart failure [C], nephropathy [D]) stratified by treatment group. For regression models (six of eight models depicted in gray), the increase in Akaike information criterion (AIC) upon removal from the full model was computed for each variable and ranked. For random forest models (two of eight models depicted in orange, including coronary artery events in the surgical patients and nephropathy in the nonsurgical patients), a permutation technique was used by comparing the out-of-bag prediction accuracy (based on the C-index) of the original forest with that after randomly permuting each variable. COPD, chronic obstructive pulmonary disease.