External validation of five predictive models for postoperative cardiopulmonary morbidity in a Chinese population receiving lung resection

Abstract

Background: No postoperative cardiopulmonary morbidity models have been developed or validated in Chinese patients with lung resection. This study aims to externally validate five predictive models, including Eurolung models, the Brunelli model and the Age-adjusted Charlson Comorbidity Index, in a Chinese population.

Methods: Patients with lung cancer who underwent anatomic lung resection between 2018/09/01 and 2019/08/31 in our center were involved. Model discrimination was assessed by the area under the receiver operating characteristic curve. Model calibration was evaluated by the Hosmer-Lemeshow test. Calibration curves were plotted. Specificity, sensitivity, negative predictive value, positive predictive value and accuracy were calculated. Model updating was achieved by re-estimating the intercept and/or the slope of the linear predictor and re-estimating all coefficients.

Results: Among 1085 patients, 91 patients had postoperative cardiopulmonary complications defined by the European Society of Thoracic Surgeons. For original models, only parsimonious Eurolung1 had acceptable discrimination (area under the receiver operating characteristic curve = 0.688, 95% confidence interval 0.630-0.745) and calibration (p = 0.23 > 0.05) abilities simultaneously. Its sensitivity, specificity, positive predictive value, negative predictive value and accuracy were 0.700, 0.649, 0.153, 0.960 and 0.653, respectively. In the secondary analysis, increased pleural effusion (n = 94), which was nonchylous and nonpurulent, was labeled as a kind of postoperative complication. The area under the receiver operating characteristic curve of the models increased slightly, but all models were miscalibrated. The original Eurolung1 model had the highest discrimination ability but poor calibration, and thus it was updated by three methods. After model updating, new models showed good calibration and small improvements in discrimination. The discrimination ability was still merely acceptable.

Conclusions: Overall, none of the models performed well on postoperative cardiopulmonary morbidity prediction in this Chinese population. The original parsimonious Eurolung1 and the updated Eurolung1 were the best-performing models on morbidity prediction, but their discrimination ability only achieved an acceptable level. A multicenter study with more relevant variables and sophisticated statistical methods is warranted to develop new models among Chinese patients in the future.

Keywords: Lung cancer; Morbidity; Predictive models; Validation.

Figure 1. The flowchart of patient selection.

Figure 2. The model performances in the first analysis.

Postoperative cardiopulmonary complications were defined by the European Society of Thoracic Surgeons. (A) The receiver operating characteristic curves (ROC) and the area under the ROC curve (AUC), indicating discrimination ability. (B) Calibration plots. 2016E1, the logit form of Eurolung1; 2019E1, the logit form of parsimonious Eurolung1; Brunelli, the Brunelli model; aE1, the aggregate form of Eurolung1; ACCI, the Age-adjusted Charlson Comorbidity Index.

Figure 3. The model performances in the secondary analysis.

Increased pleural effusion was further regarded as a kind of postoperative cardiopulmonary complication. (A) The receiver operating characteristic curves (ROC) and the area under the ROC curve (AUC), indicating discrimination ability. (B) Calibration plots. 2016E1, the logit form of Eurolung1; 2019E1, the logit form of parsimonious Eurolung1; Brunelli, the Brunelli model; aE1, the aggregate form of Eurolung1; ACCI, the Age-adjusted Charlson Comorbidity Index.