Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Apr 19:373:n736.
doi: 10.1136/bmj.n736.

The methodological quality of individual participant data meta-analysis on intervention effects: systematic review

Huan Wang  1 , Yancong Chen  1 , Yali Lin  1 , Julius Abesig  1 , Irene Xy Wu  2 , Wilson Tam  3
Affiliations

The methodological quality of individual participant data meta-analysis on intervention effects: systematic review

Huan Wang et al. BMJ. .

Abstract

Objective: To assess the methodological quality of individual participant data (IPD) meta-analysis and to identify areas for improvement.

Design: Systematic review.

Data sources: Medline, Embase, and Cochrane Database of Systematic Reviews.

Eligibility criteria for selecting studies: Systematic reviews with IPD meta-analyses of randomised controlled trials on intervention effects published in English.

Results: 323 IPD meta-analyses covering 21 clinical areas and published between 1991 and 2019 were included: 270 (84%) were non-Cochrane reviews and 269 (84%) were published in journals with a high impact factor (top quarter). The IPD meta-analyses showed low compliance in using a satisfactory technique to assess the risk of bias of the included randomised controlled trials (43%, 95% confidence interval 38% to 48%), accounting for risk of bias when interpreting results (40%, 34% to 45%), providing a list of excluded studies with justifications (32%, 27% to 37%), establishing an a priori protocol (31%, 26% to 36%), prespecifying methods for assessing both the overall effects (44%, 39% to 50%) and the participant-intervention interactions (31%, 26% to 36%), assessing and considering the potential of publication bias (31%, 26% to 36%), and conducting a comprehensive literature search (19%, 15% to 23%). Up to 126 (39%) IPD meta-analyses failed to obtain IPD from 90% or more of eligible participants or trials, among which only 60 (48%) provided reasons and 21 (17%) undertook certain strategies to account for the unavailable IPD.

Conclusions: The methodological quality of IPD meta-analyses is unsatisfactory. Future IPD meta-analyses need to establish an a priori protocol with prespecified data syntheses plan, comprehensively search the literature, critically appraise included randomised controlled trials with appropriate technique, account for risk of bias during data analyses and interpretation, and account for unavailable IPD.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from supported by the High-level Talents Introduction Plan from Central South University and the National Natural Science Foundation of China for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Figures

Fig 1
Fig 1
Screening and selection process of individual participant data (IPD) meta-analyses. RCT=randomised controlled trials
Fig 2
Fig 2
The methodological quality on six selected items of the 323 sampled individual participant data meta-analyses over time. Item 2: Did the report of the review contain an explicit statement that the review methods were established before conduct of the review and did the report justify any significant deviations from the protocol? Item 9-1: Did the review authors use a satisfactory technique for assessing the risk of bias in individual studies that were included in the review? Item 11-2: Was the choice of one stage or two stage analysis specified in advance or results for both approaches provided? Item 12: If meta-analysis was performed, did the review authors assess the potential impact of risk of bias in individual studies on the results of the meta-analysis or other evidence synthesis? Item 13: Did the review authors account for risk of bias in primary studies when interpreting or discussing the results of the review? Item 15: If they performed quantitative synthesis did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?
See this image and copyright information in PMC

References

    1. Higgins JPT, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. 10.1002/9781119536604. - DOI
    1. Tierney JF, Vale C, Riley R, et al. Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use. PLoS Med 2015;12:e1001855. 10.1371/journal.pmed.1001855 - DOI - PMC - PubMed
    1. Oxman AD, Clarke MJ, Stewart LA. From science to practice. Meta-analyses using individual patient data are needed. JAMA 1995;274:845-6. 10.1001/jama.1995.03530100085040 - DOI - PubMed
    1. Nevitt SJ, Marson AG, Davie B, Reynolds S, Williams L, Smith CT. Exploring changes over time and characteristics associated with data retrieval across individual participant data meta-analyses: systematic review. BMJ 2017;357:j1390. 10.1136/bmj.j1390 - DOI - PMC - PubMed
    1. Vale CL, Rydzewska LHM, Rovers MM, Emberson JR, Gueyffier F, Stewart LA, Cochrane IPD Meta-analysis Methods Group . Uptake of systematic reviews and meta-analyses based on individual participant data in clinical practice guidelines: descriptive study. BMJ 2015;350:h1088. 10.1136/bmj.h1088 - DOI - PMC - PubMed

Publication types