Prone position for acute respiratory distress syndrome and the hazards of meta-analysis

Abstract

Background: Researchers have tried unsuccessfully for many years using randomized controlled trials to show the efficacy of prone ventilation in treating ARDS. These failed attempts were of use in designing the successful PROSEVA trial, published in 2013. However, the evidence provided by meta-analyses in support of prone ventilation for ARDS was too low to be conclusive. The present study shows that meta-analysis is indeed not the best approach for the assessment of evidence as to the efficacy of prone ventilation.

Methods: We performed a cumulative meta-analysis to prove that only the PROSEVA trial, due to its strong protective effect, has substantially impacted on the outcome. We also replicated nine published meta-analyses including the PROSEVA trial. We performed leave-one-out analyses, removing one trial at a time from each meta-analysis, measuring p values for effect size, and also the Cochran's Q test for heterogeneity assessment. We represented these analyses in a scatter plot to identify outlier studies influencing heterogeneity or overall effect size. We used interaction tests to formally identify and evaluate differences with the PROSEVA trial.

Results: The positive effect of the PROSEVA trial accounted for most of the heterogeneity and for the reduction of overall effect size in the meta-analyses. The interaction tests we conducted on the nine meta-analyses formally confirmed the difference in the effectiveness of prone ventilation between the PROSEVA trial the other studies.

Conclusions: The clinical lack of homogeneity between the PROSEVA trial design and the other studies should have discouraged the use of meta-analysis. Statistical considerations support this hypothesis, suggesting that the PROSEVA trial is an independent source of evidence.

Keywords: ARDS; Clinical heterogeneity; Meta-analysis; Statistical heterogeneity.

Fig. 1

cumulative meta-analysis considering short-term mortality.

Fig. 2

cumulative meta-analysis considering long term mortality.

Fig. 3

The plot depicts the degree of correlation between p values for effect size and p values for heterogeneity. Points within the dashed square, are the meta-analyses performed leaving out the PROSEVA trial by Guerin et al. In all the other cases (points within the dashed triangle), the trial is included and the other RCTs are left out one at the time. When the PROSEVA trial is left out from the meta-analysis, both p values increase significantly compared to when the other trials are left out. This means that both the p values for both the overall effect and heterogeneity are strongly influenced by the presence of the PROSEVA trial, which should be regarded as the only outlier. CMAJ = Canadian Medical Association Journal, ATS = American Thoracic Society, Crit Care = Critical Care, Crit Care Med = Critical Care Medicine, J Thorac Dis = Journal of Thoracic Diseases, Cochrane = Cochrane Database of Systematic Reviews, Med Int = Medicina Intensiva, Int Care Med = Intensive Care Medicine.