Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Dec 6:4:267.
doi: 10.3389/fpubh.2016.00267. eCollection 2016.

Evaluating and Using Observational Evidence: The Contrasting Views of Policy Makers and Epidemiologists

Lily O'Donoughue Jenkins  1 , Paul M Kelly  2 , Nicolas Cherbuin  1 , Kaarin J Anstey  1
Affiliations

Evaluating and Using Observational Evidence: The Contrasting Views of Policy Makers and Epidemiologists

Lily O'Donoughue Jenkins et al. Front Public Health. .

Abstract

Background: Currently, little is known about the types of evidence used by policy makers. This study aimed to investigate how policy makers in the health domain use and evaluate evidence and how this differs from academic epidemiologists. By having a better understanding of how policy makers select, evaluate, and use evidence, academics can tailor the way in which that evidence is produced, potentially leading to more effective knowledge translation.

Methods: An exploratory mixed-methods study design was used. Quantitative measures were collected via an anonymous online survey (n = 28), with sampling from three health-related government and non-government organizations. Semi-structured interviews with policy makers (n = 20) and epidemiologists (n = 6) were conducted to gather qualitative data.

Results: Policy makers indicated systematic reviews were the preferred research resource (19%), followed closely by qualitative research (16%). Neither policy makers nor epidemiologists used grading instruments to evaluate evidence. In the web survey, policy makers reported that consistency and strength of evidence (93%), the quality of data (93%), bias in the evidence (79%), and recency of evidence (79%) were the most important factors taken into consideration when evaluating the available evidence. The same results were found in the qualitative interviews. Epidemiologists focused on the methodology used in the study. The most cited barriers to using robust evidence, according to policy makers, were political considerations (60%), time limitations (55%), funding (50%), and research not being applicable to current policies (50%).

Conclusion: The policy maker's investigation did not report a systematic approach to evaluating evidence. Although there was some overlap between what policy makers and epidemiologists identified as high-quality evidence, there was also some important differences. This suggests that the best scientific evidence may not routinely be used in the development of policy. In essence, the policy-making process relied on other jurisdictions' policies and the opinions of internal staff members as primary evidence sources to inform policy decisions. Findings of this study suggest that efforts should be directed toward making scientific information more systematically available to policy makers.

Keywords: evidence-based practice; government; knowledge translation; mixed-methods research; policy making.

PubMed Disclaimer

Figures

Figure 1
Figure 1
The most important factors taken into consideration when evaluating evidence.
Figure 2
Figure 2
The relative quality of specific research methods and data synthesis techniques.
See this image and copyright information in PMC

References

    1. Zardo P, Collie A. Measuring use of research evidence in public health policy: a policy content analysis. BMC Public Health (2014) 14(1):496.10.1186/1471-2458-14-496 - DOI - PMC - PubMed
    1. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof (2006) 26(1):13–24.10.1002/chp.47 - DOI - PubMed
    1. Jacobson N, Butterill D, Goering P. Organizational factors that influence university-based researchers’ engagement in knowledge transfer activities. Sci Commun (2004) 25(3):246–59.10.1177/1075547003262038 - DOI
    1. Cvitanovic C, Hobday AJ, van Kerkhoff L, Wilson SK, Dobbs K, Marshall NA. Improving knowledge exchange among scientists and decision-makers to facilitate the adaptive governance of marine resources: a review of knowledge and research needs. Ocean Coastal Manag (2015) 112:25–35.10.1016/j.ocecoaman.2015.05.002 - DOI
    1. Bowen S, Zwi A, Sainsbury P. What evidence informs government population health policy? Lessons from early childhood intervention policy in Australia. N S W Public Health Bull (2005) 16(11–12):180.10.1071/NB05050 - DOI - PubMed

LinkOut - more resources