Bayesian model-averaged meta-analysis in medicine

Abstract

We outline a Bayesian model-averaged (BMA) meta-analysis for standardized mean differences in order to quantify evidence for both treatment effectiveness $\delta$ and across-study heterogeneity $\tau$ . We construct four competing models by orthogonally combining two present-absent assumptions, one for the treatment effect and one for across-study heterogeneity. To inform the choice of prior distributions for the model parameters, we used 50% of the Cochrane Database of Systematic Reviews to specify rival prior distributions for $\delta$ and $\tau$ . The relative predictive performance of the competing models and rival prior distributions was assessed using the remaining 50% of the Cochrane Database. On average, ${\mathscr{H}}_1^r$ -the model that assumes the presence of a treatment effect as well as across-study heterogeneity-outpredicted the other models, but not by a large margin. Within ${\mathscr{H}}_1^r$ , predictive adequacy was relatively constant across the rival prior distributions. We propose specific empirical prior distributions, both for the field in general and for each of 46 specific medical subdisciplines. An example from oral health demonstrates how the proposed prior distributions can be used to conduct a BMA meta-analysis in the open-source software R and JASP. The preregistered analysis plan is available at https://osf.io/zs3df/.

Keywords: Bayes factor; empirical prior distribution; evidence.

FIGURE 1

Flowchart of the study selection procedure and data processing steps for the training data set (left) and the test data set (right) [Colour figure can be viewed at wileyonlinelibrary.com]

FIGURE 2

Frequentist effect sizes estimates and candidate prior distributions from the training data set. Histogram and tick marks display the estimated effect size estimates (left) and between‐study standard deviation estimates (right), whereas lines represent three associated candidate prior distributions for the population effect size parameter $\delta$ (left) and four candidate prior distributions for the population between‐study standard deviation $\tau$ (right; see Table 1). Twelve effect sizes outside of the $\pm 1.5$ range are not shown and twenty‐four $\tau$ estimates larger than 1 and sixty‐eight $\tau$ estimates lower than 0.01 are not shown

FIGURE 3

Average posterior probabilities (AV. PoMP) for each of the 12 prior configurations under ${\mathscr{H}}_1^r$ for all 2,406 test‐set comparisons individually. For each comparison, the color gradient ranges from white (low posterior probability) to dark red (high posterior probability). The numbers in parentheses are the averaged posterior probabilities across all 2,406 comparisons (conditional on ${\mathscr{H}}_1^r$). The prior probability for each configuration is $1/12\approx 0.083$. See also Table 2 [Colour figure can be viewed at wileyonlinelibrary.com]

FIGURE 4

Model‐averaged posterior probabilities (Av. PoMP) for each of the four model types for all 2,406 test‐set comparisons individually (left) and each prior distribution for all 2,406 test‐set comparisons individually (right). For each comparison, the color gradient ranges from white (low posterior probability) to dark red (high posterior probability). The numbers in parentheses are the averaged posterior probabilities across all 2,406 comparisons. In the left panel, the prior probability for each model type is $1/4$, see also Table 4. In the right panel, the prior probability is $1/3$ for each prior distribution on $\delta$, and $1/4$ for each prior distribution on $\tau$, see also Table 5 [Colour figure can be viewed at wileyonlinelibrary.com]

FIGURE 5

Inclusion Bayes factors in favor of the presence of a treatment effect (left) and in favor of the presence of across‐study heterogeneity (right) for the 2,406 comparisons in the test set. Not shown are log Bayes factors that exceed 21: twelve log Bayes factors for the presence of a treatment effect and 255 log Bayes factors for the presence of heterogeneity, or are lower than ‐3: one log Bayes factors for the presence of a treatment effect and four log Bayes factors for the presence of heterogeneity

FIGURE 6

Subfield‐specific prior distributions for parameter $\delta$ (left panel) and parameter $\tau$ (right panel) for 46 individual topics from the Cochrane Database of Systematic Reviews estimated by hierarchical regression based on the complete data set. See also Table 6

FIGURE 7

JASP screenshot of a Bayesian model‐averaged meta‐analysis of the Poulsen et al comparison concerning the effect of potassium‐containing toothpaste on dentine tactile hypersensitivity. The left input panel shows the specification of the “Oral Health” CDSR subfield‐specific prior distributions for effect size $\delta$ and heterogeneity $\tau$. The right output panel shows the corresponding results [Colour figure can be viewed at wileyonlinelibrary.com]