Predicting responders to prone positioning in mechanically ventilated patients with COVID-19 using machine learning

Abstract

Background: For mechanically ventilated critically ill COVID-19 patients, prone positioning has quickly become an important treatment strategy, however, prone positioning is labor intensive and comes with potential adverse effects. Therefore, identifying which critically ill intubated COVID-19 patients will benefit may help allocate labor resources.

Methods: From the multi-center Dutch Data Warehouse of COVID-19 ICU patients from 25 hospitals, we selected all 3619 episodes of prone positioning in 1142 invasively mechanically ventilated patients. We excluded episodes longer than 24 h. Berlin ARDS criteria were not formally documented. We used supervised machine learning algorithms Logistic Regression, Random Forest, Naive Bayes, K-Nearest Neighbors, Support Vector Machine and Extreme Gradient Boosting on readily available and clinically relevant features to predict success of prone positioning after 4 h (window of 1 to 7 h) based on various possible outcomes. These outcomes were defined as improvements of at least 10% in PaO₂/FiO₂ ratio, ventilatory ratio, respiratory system compliance, or mechanical power. Separate models were created for each of these outcomes. Re-supination within 4 h after pronation was labeled as failure. We also developed models using a 20 mmHg improvement cut-off for PaO₂/FiO₂ ratio and using a combined outcome parameter. For all models, we evaluated feature importance expressed as contribution to predictive performance based on their relative ranking.

Results: The median duration of prone episodes was 17 h (11-20, median and IQR, N = 2632). Despite extensive modeling using a plethora of machine learning techniques and a large number of potentially clinically relevant features, discrimination between responders and non-responders remained poor with an area under the receiver operator characteristic curve of 0.62 for PaO₂/FiO₂ ratio using Logistic Regression, Random Forest and XGBoost. Feature importance was inconsistent between models for different outcomes. Notably, not even being a previous responder to prone positioning, or PEEP-levels before prone positioning, provided any meaningful contribution to predicting a successful next proning episode.

Conclusions: In mechanically ventilated COVID-19 patients, predicting the success of prone positioning using clinically relevant and readily available parameters from electronic health records is currently not feasible. Given the current evidence base, a liberal approach to proning in all patients with severe COVID-19 ARDS is therefore justified and in particular regardless of previous results of proning.

Keywords: Acute respiratory distress syndrome; COVID-19; Mechanical ventilation.

Fig. 1

Model performance by ROC AUC score for predicting improvement in various outcome parameters after turning patients to a prone position. The ROC AUC compares the true positive rate to the false positive rate where a performance of 1.0 reflects perfect scores where 0.5 describes complete randomness. LR logistic regression, RF  random forest, KNN  K-Nearest Neighbors, SVM  support vector machine, GNB Gaussian Naïve Bayes, XGB  eXtreme Gradient Boosting

Fig. 2

Model performance by F1-score for predicting improvement in various outcome parameters after turning patients to a prone position. The F1-score combines the precision (positive predictive value) and recall (sensitivity) scores to provide a single metric to compare model performance where a performance of 1.0 reflects perfect scores while 0.0 reflects the worst performance. LR logistic regression, RF  random forest, KNN  K-Nearest Neighbors, SVM  support vector machine, GNB Gaussian Naïve Bayes, XGB  eXtreme Gradient Boosting