The albatross plot: A novel graphical tool for presenting results of diversely reported studies in a systematic review

Abstract

Meta-analyses combine the results of multiple studies of a common question. Approaches based on effect size estimates from each study are generally regarded as the most informative. However, these methods can only be used if comparable effect sizes can be computed from each study, and this may not be the case due to variation in how the studies were done or limitations in how their results were reported. Other methods, such as vote counting, are then used to summarize the results of these studies, but most of these methods are limited in that they do not provide any indication of the magnitude of effect. We propose a novel plot, the albatross plot, which requires only a 1-sided P value and a total sample size from each study (or equivalently a 2-sided P value, direction of effect and total sample size). The plot allows an approximate examination of underlying effect sizes and the potential to identify sources of heterogeneity across studies. This is achieved by drawing contours showing the range of effect sizes that might lead to each P value for given sample sizes, under simple study designs. We provide examples of albatross plots using data from previous meta-analyses, allowing for comparison of results, and an example from when a meta-analysis was not possible.

Keywords: evidence synthesis; graphical tool; methodology; systematic review.

Figure 1

Albatross plot for studies of the effect of exercise training on left ventricular fraction after acute myocardial infarction, with contours for standardized mean differences (SMDs), using data from Zhang et al11

Figure 2

Albatross plot for studies of student ratings of their college instructors and student achievement levels with contours for correlation coefficients, using data from Becker14

Figure 3

Albatross plot for studies of the association between milk intake an insulin‐like growth factor‐I, using data from Harrison et al (Harrison et al, In press meta‐analysis, 2016)15

Figure 4

Albatross plot for the association between body mass index and prostate specific antigen, using data from Harrison et al (Harrison et al, unpublished meta‐analysis, 2016)16

Figure 5

Forest plot of studies of the effect of exercise training on left ventricular fraction after acute myocardial infarction, data from Zhang et al.11 I‐V subtotals represent fixed effect meta‐analyses; D+L subtotals represent random effect meta‐analyses. I‐squared is a relative measure of heterogeneity in relation to total variability within each subgroup. SMD, standardized mean difference