Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Nov 23;19(1):279.
doi: 10.1186/s12916-021-02151-w.

Risk of bias in observational studies using routinely collected data of comparative effectiveness research: a meta-research study

Van Thu Nguyen  1   2 , Mishelle Engleton  3 , Mauricia Davison  3   4 , Philippe Ravaud  3   4   5 , Raphael Porcher  3   4   5 , Isabelle Boutron  3   4   5
Affiliations

Risk of bias in observational studies using routinely collected data of comparative effectiveness research: a meta-research study

Van Thu Nguyen et al. BMC Med. .

Abstract

Background: To assess the completeness of reporting, research transparency practices, and risk of selection and immortal bias in observational studies using routinely collected data for comparative effectiveness research.

Method: We performed a meta-research study by searching PubMed for comparative effectiveness observational studies evaluating therapeutic interventions using routinely collected data published in high impact factor journals from 01/06/2018 to 30/06/2020. We assessed the reporting of the study design (i.e., eligibility, treatment assignment, and the start of follow-up). The risk of selection bias and immortal time bias was determined by assessing if the time of eligibility, the treatment assignment, and the start of follow-up were synchronized to mimic the randomization following the target trial emulation framework.

Result: Seventy-seven articles were identified. Most studies evaluated pharmacological treatments (69%) with a median sample size of 24,000 individuals. In total, 20% of articles inadequately reported essential information of the study design. One-third of the articles (n = 25, 33%) raised some concerns because of unclear reporting (n = 6, 8%) or were at high risk of selection bias and/or immortal time bias (n = 19, 25%). Only five articles (25%) described a solution to mitigate these biases. Six articles (31%) discussed these biases in the limitations section.

Conclusion: Reporting of essential information of study design in observational studies remained suboptimal. Selection bias and immortal time bias were common methodological issues that researchers and physicians should be aware of when interpreting the results of observational studies using routinely collected data.

Keywords: Emulated trial; Meta-research; Observational studies; Risk of bias; Routinely collected data.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Study selection process
Fig. 2
Fig. 2
The number of studies at risk of bias due to lack of synchronization. Nineteen (25%) studies had a high risk of bias due to the lack of synchronization. Of these, 14 proposed no solution, and 5 used inadequate methods to address the bias. Six studies inadequately reported to enable the assessment of synchronization. Fifty-two (68%) studies had low risk of bias
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Stuart EA, Ackerman B, Westreich D. Generalizability of randomized trial results to target populations: design and analysis possibilities. Res Soc Work Pract. 2017;28(5):532–537. doi: 10.1177/1049731517720730. - DOI - PMC - PubMed
    1. McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomized controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7(1):9. doi: 10.1186/1745-6215-7-9. - DOI - PMC - PubMed
    1. Hernan M, Robins JM. Causal inference: what if. Boca Raton: Chapman & Hall/CRC; 2020.
    1. Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183(8):758–764. doi: 10.1093/aje/kwv254. - DOI - PMC - PubMed
    1. Gershman B, Guo DP, Dahabreh IJ. Using observational data for personalized medicine when clinical trial evidence is limited. Fertil Steril. 2018;109(6):946–951. doi: 10.1016/j.fertnstert.2018.04.005. - DOI - PubMed

LinkOut - more resources