Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Mar;84(3):419-433.
doi: 10.1111/bcp.13469. Epub 2017 Dec 20.

Measuring the impact of medicines regulatory interventions - Systematic review and methodological considerations

Thomas Goedecke  1 , Daniel R Morales  1   2 , Alexandra Pacurariu  1   3 , Xavier Kurz  1
Affiliations

Measuring the impact of medicines regulatory interventions - Systematic review and methodological considerations

Thomas Goedecke et al. Br J Clin Pharmacol. 2018 Mar.

Abstract

Aims: Evaluating the public health impact of regulatory interventions is important but there is currently no common methodological approach to guide this evaluation. This systematic review provides a descriptive overview of the analytical methods for impact research.

Methods: We searched MEDLINE and EMBASE for articles with an empirical analysis evaluating the impact of European Union or non-European Union regulatory actions to safeguard public health published until March 2017. References from systematic reviews and articles from other known sources were added. Regulatory interventions, data sources, outcomes of interest, methodology and key findings were extracted.

Results: From 1246 screened articles, 229 were eligible for full-text review and 153 articles in English language were included in the descriptive analysis. Over a third of articles studied analgesics and antidepressants. Interventions most frequently evaluated are regulatory safety communications (28.8%), black box warnings (23.5%) and direct healthcare professional communications (10.5%); 55% of studies measured changes in drug utilization patterns, 27% evaluated health outcomes, and 18% targeted knowledge, behaviour or changes in clinical practice. Unintended consequences like switching therapies or spill-over effects were rarely evaluated. Two-thirds used before-after time series and 15.7% before-after cross-sectional study designs. Various analytical approaches were applied including interrupted time series regression (31.4%), simple descriptive analysis (28.8%) and descriptive analysis with significance tests (23.5%).

Conclusion: Whilst impact evaluation of pharmacovigilance and product-specific regulatory interventions is increasing, the marked heterogeneity in study conduct and reporting highlights the need for scientific guidance to ensure robust methodologies are applied and systematic dissemination of results occurs.

Keywords: analytic method; before-after study design; impact evaluation; interrupted time series; pharmacovigilance; real-world effectiveness; regulatory interventions.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Literature search and systematic review strategy. #Known literature and relevant references of published systematic reviews were included. *Duplicates, abstracts, letters to editors, commentaries and articles analysing the impact of other interventions (i.e. process and health policy related) were excluded
Figure 2
Figure 2
Types of data sources used for regulatory impact research (n = 153 articles)
Figure 3
Figure 3
Distribution of study designs (A) and analytical methods (B) in impact research over time (n = 153). *Includes randomized clinical trials and cohort studies. #P = 0.003 using chi‐squared test for trend
See this image and copyright information in PMC

Comment in

References

    1. Bouvy JC, De Bruin ML, Koopmanschap MA. Epidemiology of adverse drug reactions in Europe: a review of recent observational studies. Drug Saf 2015; 38: 437–453. - PMC - PubMed
    1. Arlett P, Portier G, de Lisa R, Blake K, Wathion N, Dogne JM, et al Proactively managing the risk of marketed drugs: experience with the EMA Pharmacovigilance risk assessment committee. Nat Rev Drug Discov 2014; 13: 395–397. - PubMed
    1. Pharmacovigilance Risk Assessment Committee . PRAC strategy on measuring the impact of pharmacovigilance activities (Rev 1) (EMA/165407/2017). 2017. Available at http://www.ema.europa.eu/docs/en_GB/document_library/Other/2016/01/WC500... (last accessed 28 July 2017).
    1. Coglianese C. Measuring regulatory performance. Evaluating the impact of regulation and policy. Organisation for Economic Co‐operation and Development (OECD). Expert Paper No. 1; August 2012. Available from: http://www.oecd.org/gov/regulatory-policy/1_coglianese%20web.pdf (last accessed 28 July 2017).
    1. Laporte JR. Fifty years of pharmacovigilance ‐ medicines safety and public health. Pharmacoepidemiol Drug Saf 2016; 25: 725–732. - PubMed

Publication types