Validation and utility of ARDS subphenotypes identified by machine-learning models using clinical data: an observational, multicohort, retrospective analysis

Abstract

Background: Two acute respiratory distress syndrome (ARDS) subphenotypes (hyperinflammatory and hypoinflammatory) with distinct clinical and biological features and differential treatment responses have been identified using latent class analysis (LCA) in seven individual cohorts. To facilitate bedside identification of subphenotypes, clinical classifier models using readily available clinical variables have been described in four randomised controlled trials. We aimed to assess the performance of these models in observational cohorts of ARDS.

Methods: In this observational, multicohort, retrospective study, we validated two machine-learning clinical classifier models for assigning ARDS subphenotypes in two observational cohorts of patients with ARDS: Early Assessment of Renal and Lung Injury (EARLI; n=335) and Validating Acute Lung Injury Markers for Diagnosis (VALID; n=452), with LCA-derived subphenotypes as the gold standard. The primary model comprised only vital signs and laboratory variables, and the secondary model comprised all predictors in the primary model, with the addition of ventilatory variables and demographics. Model performance was assessed by calculating the area under the receiver operating characteristic curve (AUC) and calibration plots, and assigning subphenotypes using a probability cutoff value of 0·5 to determine sensitivity, specificity, and accuracy of the assignments. We also assessed the performance of the primary model in EARLI using data automatically extracted from an electronic health record (EHR; EHR-derived EARLI cohort). In Large Observational Study to Understand the Global Impact of Severe Acute Respiratory Failure (LUNG SAFE; n=2813), a multinational, observational ARDS cohort, we applied a custom classifier model (with fewer variables than the primary model) to determine the prognostic value of the subphenotypes and tested their interaction with the positive end-expiratory pressure (PEEP) strategy, with 90-day mortality as the dependent variable.

Findings: The primary clinical classifier model had an area under receiver operating characteristic curve (AUC) of 0·92 (95% CI 0·90-0·95) in EARLI and 0·88 (0·84-0·91) in VALID. Performance of the primary model was similar when using exclusively EHR-derived predictors compared with manually curated predictors (AUC=0·88 [95% CI 0·81-0·94] vs 0·92 [0·88-0·97]). In LUNG SAFE, 90-day mortality was higher in patients assigned the hyperinflammatory subphenotype than in those with the hypoinflammatory phenotype (414 [57%] of 725 vs 694 [33%] of 2088; p<0·0001). There was a significant treatment interaction with PEEP strategy and ARDS subphenotype (p=0·041), with lower 90-day mortality in the high PEEP group of patients with the hyperinflammatory subphenotype (hyperinflammatory subphenotype: 169 [54%] of 313 patients in the high PEEP group vs 127 [62%] of 205 patients in the low PEEP group; hypoinflammatory subphenotype: 231 [34%] of 675 patients in the high PEEP group vs 233 [32%] of 734 patients in the low PEEP group).

Interpretation: Classifier models using clinical variables alone can accurately assign ARDS subphenotypes in observational cohorts. Application of these models can provide valuable prognostic information and could inform management strategies for personalised treatment, including application of PEEP, once prospectively validated.

Funding: US National Institutes of Health and European Society of Intensive Care Medicine.

Figure 1. Schematic of analysis plan.

The models were originally trained in ARMA, ALVEOLI and FACTT (n = 2022) and tested in SAILS (n=745), which were all randomised controlled trials. The models were validated in two observational cohorts: EARLI (n=335) and VALID (n=452). A custom (“LUNG SAFE”) model using a limited set of predictor variables was developed to evaluate the clinical utility of ARDS subphenotypes in LUNG SAFE (n=2813) a large multinational observational cohort of ARDS. The optimal probability cutoff for the “LUNG SAFE” model determined by first evaluating the model in VALID (n=452).

Figure 2. Receiver operating characteristic (ROC) curve for primary (“vital and labs”) model in EARLI (n=335) and VALID (n=452).

AUC = Area under the ROC curve. EARLI AUC = 0.92; VALID AUC = 0.88.

Figure 3. Differences in protein biomarkers in ARDS subphenotypes.

ARDS subphenotypes were identified by the “vitals and labs” model using a probability cut-off of 0.5. Differences in biomarker data are presented in EARLI (n=335) and VALID (n=452). Y-axis was limited to aid better data visualization. Consequently, in EARLI, 9, 10, 13, and 4 observations were censored, and in VALID, 13, 16, 17, and 3 observations were censored for Interleukin-6, Interleukin-8, Soluble tumor necrosis factor (TNF) receptor-1, and Protein C, respectively.

Figure 4. Survival curves of the two ARDS subphenotypes in LUNG SAFE (n=2813).

ARDS subphenotypes were identified by a custom clinical-classifier (“LUNG SAFE”) model using a probability cutoff of 0·4 to assign class. Abbreviations: Acute Respiratory Distress Syndrome (ARDS). P-value was calculated using the log-rank test.

Figure 5. Comparison of respiratory variables between the two ARDS subphenotypes in LUNG SAFE (n=2813).

ARDS subphenotypes were identified by a custom clinical-classifier (“LUNG SAFE”) model using a probability cutoff of 0·4 to assign class. Driving pressure is defined as the difference between plateau pressure and PEEP. Abbreviations: Positive End Expiratory Pressure (PEEP); Hyperinflammatory subphenotype (Hyper); Hypoinflammatory subphenotype (Hypo). P-value was calculated using either the t-test or Wilcoxon rank test depending on the distribution of the data.