Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2018 Dec;16(6):819-836.
doi: 10.1007/s40258-018-0415-5.

Use of Expert Judgement Across NICE Guidance-Making Programmes: A Review of Current Processes and Suitability of Existing Tools to Support the Use of Expert Elicitation

Alison Peel  1 , Michelle Jenks  2 , Moni Choudhury  3 , Rosemary Lovett  3 , Juan Carlos Rejon-Parrilla  4 , Andrew Sims  5   6 , Joyce Craig  2
Affiliations
Review

Use of Expert Judgement Across NICE Guidance-Making Programmes: A Review of Current Processes and Suitability of Existing Tools to Support the Use of Expert Elicitation

Alison Peel et al. Appl Health Econ Health Policy. 2018 Dec.

Erratum in

Abstract

Objectives: This study aimed to review current use of experts within National Institute for Health and Care Excellence (NICE) guidance-making programmes, identify improvements in use of expert judgement, and to assess tools and protocols to support the elicitation of information from experts for use by NICE.

Methods: The study comprised a review of NICE process guides; semi-structured interviews with individuals representing each NICE guidance-making programme and a comparison of the suitability of currently available tools and protocols for expert elicitation to the requirements of NICE programmes.

Results: Information elicited from experts and the way in which it is used varies across NICE guidance-making programmes. Experts' involvement can be as intensive as being a member of a committee and thus having direct influence on recommendations or limited one-off consultations on specific parameters. We identified 16 tools for expert elicitation that were potentially relevant. None fully met the requirements of NICE, although an existing tool could be potentially adapted. Ongoing research to develop a reference protocol for expert elicitation in healthcare decision making may be of use in future.

Conclusions: NICE uses expert judgement across all its guidance-making programmes, but its uses vary considerably. There is no currently available tool for expert elicitation suitable for use by NICE. However, adaptation of an existing tool or ongoing research in the area could address this in the future.

PubMed Disclaimer

Conflict of interest statement

Conflict of interest

Newcastle upon Tyne Hospitals and York Health Economics Consortium are funded by NICE to support the Medical Technologies Evaluation Programme as an External Assessment Centre (EAC). Four of the authors (AP, MJ, JC, AS) work for the EAC but otherwise have no conflicts of interest. MC, JCRP and RL are employees of NICE.

Figures

Box 1
Box 1
Definitions of expert elicitation and expert opinion used in this study [1]
Box 2
Box 2
Use of expert opinion at NICE
See this image and copyright information in PMC

References

    1. Iglesias CP, Thompson A, Rogowski WH, Payne K. Reporting guidelines for the use of expert judgement in model-based economic evaluations. PharmacoEconomics. 2016;34(11):1161–1172. doi: 10.1007/s40273-016-0425-9. - DOI - PubMed
    1. Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. J Clin Nurs. 2003;12(1):77–84. doi: 10.1046/j.1365-2702.2003.00662.x. - DOI - PubMed
    1. O’Hagan A, Oakley J. SHELF: the Sheffield Elicitation Framework v3.0. 2016. Available from: http://www.tonyohagan.co.uk/shelf/.
    1. O’Hagan A, Buck CE, Daneshkhah AJ, Eiser R, Garthwaite PH, Jenkinson DJ, et al. Uncertain judgements: eliciting experts’ probabilities. New York: Wiley; 2006.
    1. Garthwaite PH, Kadane JB, O’Hagan A. Statistical methods for eliciting probability distributions. J Am Stat Assoc. 2005;100(470):680–701. doi: 10.1198/016214505000000105. - DOI

MeSH terms

LinkOut - more resources