Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Mar;17(2):143-56.
doi: 10.1016/j.jval.2013.12.011.

A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report

Marc L Berger  1 , Bradley C Martin  2 , Don Husereau  3 , Karen Worley  4 , J Daniel Allen  5 , Winnie Yang  6 , Nicole C Quon  7 , C Daniel Mullins  8 , Kristijan H Kahler  9 , William Crown  10
Affiliations

A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report

Marc L Berger et al. Value Health. 2014 Mar.

Erratum in

  • Value Health. 2014 Jun;17(4):489

Abstract

Evidence-based health care decisions are best informed by comparisons of all relevant interventions used to treat conditions in specific patient populations. Observational studies are being performed to help fill evidence gaps. Widespread adoption of evidence from observational studies, however, has been limited because of various factors, including the lack of consensus regarding accepted principles for their evaluation and interpretation. Two task forces were formed to develop questionnaires to assist decision makers in evaluating observational studies, with one Task Force addressing retrospective research and the other Task Force addressing prospective research. The intent was to promote a structured approach to reduce the potential for subjective interpretation of evidence and drive consistency in decision making. Separately developed questionnaires were combined into a single questionnaire consisting of 33 items. These were divided into two domains: relevance and credibility. Relevance addresses the extent to which findings, if accurate, apply to the setting of interest to the decision maker. Credibility addresses the extent to which the study findings accurately answer the study question. The questionnaire provides a guide for assessing the degree of confidence that should be placed from observational studies and promotes awareness of the subtleties involved in evaluating those.

Keywords: bias; checklist; comparative effectiveness research; confounding; consensus; credibility; decision making; prospective observational study; quality; questionnaire; relevance; retrospective observational study; validity.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest

Two questions were used in the conflicts of interest domain in all of the questionnaires:

Were there any potential conflicts of interest?

Conflicts of interest may be stated by authors; however, reviewers may also seek information from public sources including web-based CVs or faculty pages. In some cases, conflicts are not stated in a research report simply due to editorial policy rather than a lack of their existence. While some journals adhere strictly to uniform standards for disclosing conflicts, such as those promoted by the International Committee of Medical Journal Editors (ICMJE), others may not. Readers should not misinterpret absence of a stated conflict as evidence of absence of a conflict.

If there were potential conflicts of interest, were steps taken to address these?

Potential conflicts of interest may include financial interests in the study results, desire for professional recognition, or other non-monetary incentives. Steps to address potential conflicts of interest include disclosure of any potential conflicts of interest, involving third parties in the design, conduct, and analysis of studies, and agreements that provide independence of researchers (including freedom to publicly disseminate results) from funding entities [28].

Figures

Figure 1
Figure 1. Summary Flowchart for Observational study assessment questionnaire
Red thumbs down icons indicate that a “weakness” had been detected in one of the elements that support credibility. Red skull & cross-bones icons indicate that a potential “fatal flaw” had been detected.
See this image and copyright information in PMC

References

    1. Garrison LP, Neumann PJ, Jr, Erickson P, Marshall D, Mullins CD. Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2007;10(5):326–335. - PubMed
    1. Brixner DI, Holtorf AP, Neumann PJ, Malone DC, Watkins JB. Standardizing quality assessment of observational studies for decision making in health care. Journal of managed care pharmacy : JMCP. 2009;15(3):275–283. - PMC - PubMed
    1. Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of clinical epidemiology. 2011;64(4):401–406. - PubMed
    1. Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC health services research. 2004;4(1):38. - PMC - PubMed
    1. Glasziou P, Vandenbroucke JP, Chalmers I. Assessing the quality of research. BMJ (Clinical research ed.) 2004;328(7430):39–41. - PMC - PubMed

Publication types

LinkOut - more resources