Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Sep 2;19(1):183.
doi: 10.1186/s12874-019-0817-6.

Statistical approaches to identify subgroups in meta-analysis of individual participant data: a simulation study

Michail Belias  1 , Maroeska M Rovers  2 , Johannes B Reitsma  3   4 , Thomas P A Debray  3   4 , Joanna IntHout  2
Affiliations

Statistical approaches to identify subgroups in meta-analysis of individual participant data: a simulation study

Michail Belias et al. BMC Med Res Methodol. .

Abstract

Background: Individual participant data meta-analysis (IPD-MA) is considered the gold standard for investigating subgroup effects. Frequently used regression-based approaches to detect subgroups in IPD-MA are: meta-regression, per-subgroup meta-analysis (PS-MA), meta-analysis of interaction terms (MA-IT), naive one-stage IPD-MA (ignoring potential study-level confounding), and centred one-stage IPD-MA (accounting for potential study-level confounding). Clear guidance on the analyses is lacking and clinical researchers may use approaches with suboptimal efficiency to investigate subgroup effects in an IPD setting. Therefore, our aim is to overview and compare the aforementioned methods, and provide recommendations over which should be preferred.

Methods: We conducted a simulation study where we generated IPD of randomised trials and varied the magnitude of subgroup effect (0, 25, 50% relative reduction), between-study treatment effect heterogeneity (none, medium, large), ecological bias (none, quantitative, qualitative), sample size (50,100,200), and number of trials (5,10) for binary, continuous and time-to-event outcomes. For each scenario, we assessed the power, false positive rate (FPR) and bias of aforementioned five approaches.

Results: Naive and centred IPD-MA yielded the highest power, whilst preserving acceptable FPR around the nominal 5% in all scenarios. Centred IPD-MA showed slightly less biased estimates than naïve IPD-MA. Similar results were obtained for MA-IT, except when analysing binary outcomes (where it yielded less power and FPR < 5%). PS-MA showed similar power as MA-IT in non-heterogeneous scenarios, but power collapsed as heterogeneity increased, and decreased even more in the presence of ecological bias. PS-MA suffered from too high FPRs in non-heterogeneous settings and showed biased estimates in all scenarios. Meta-regression showed poor power (< 20%) in all scenarios and completely biased results in settings with qualitative ecological bias.

Conclusions: Our results indicate that subgroup detection in IPD-MA requires careful modelling. Naive and centred IPD-MA performed equally well, but due to less bias of the estimates in the presence of ecological bias, we recommend the latter.

Keywords: Comparison; Ecological bias; Effect modification; Heterogeneity; Individual participant data; Meta-analysis; Simulation; Statistical approaches; Subgroups.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no competing interests associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

Figures

Fig. 1
Fig. 1
Overview of simulations approximately here
Fig. 2
Fig. 2
Type I errors for the scenarios of 5 studies with 100 participants each approximately here
Fig. 3
Fig. 3
Power to detect a large subgroup effect in the scenarios of 5 studies with 100 participants each approximately here
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Chalmers I. The Cochrane collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care. Ann N Y Acad Sci. 1993;703:156–163. doi: 10.1111/j.1749-6632.1993.tb26345.x. - DOI - PubMed
    1. Debray TP, Moons KG, van Valkenhoef G, Efthimiou O, Hummel N, Groenwold RH, et al. Get real in individual participant data (IPD) meta-analysis: a review of the methodology. Res Synth Methods. 2015;6(4):293–309. doi: 10.1002/jrsm.1160. - DOI - PMC - PubMed
    1. Tierney JF, Vale C, Riley R, Smith CT, Stewart L, Clarke M, et al. Individual participant data (IPD) Meta-analyses of randomised controlled trials: guidance on their use. PLoS Med. 2015;12(7):e1001855. doi: 10.1371/journal.pmed.1001855. - DOI - PMC - PubMed
    1. Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof. 2002;25(1):76–97. doi: 10.1177/0163278702025001006. - DOI - PubMed
    1. Simmonds M, Stewart G, Stewart L. A decade of individual participant data meta-analyses: a review of current practice. Contemp Clin Trials. 2015;45:76–83. doi: 10.1016/j.cct.2015.06.012. - DOI - PubMed

Publication types

LinkOut - more resources