Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art

Abstract

Background: Multi-criteria decision analysis (MCDA) concepts, models and tools have been used increasingly in health technology assessment (HTA), with several studies pointing out practical and theoretical issues related to its use. This study provides a critical review of published studies on MCDA in the context of HTA by assessing their methodological quality and summarising methodological challenges.

Methods: A systematic review was conducted to identify studies discussing, developing or reviewing the use of MCDA in HTA using aggregation approaches. Studies were classified according to publication time and type, country of study, technology type and study type. The PROACTIVE-S approach was constructed and used to analyse methodological quality. Challenges and limitations reported in eligible studies were collected and summarised; this was followed by a critical discussion on research requirements to address the identified challenges.

Results: 129 journal articles were eligible for review, 56% of which were published in 2015-2017; 42% focused on pharmaceuticals; 36, 26 and 18% reported model applications, issues regarding MCDA implementation analyses, and proposing frameworks, respectively. Poor compliance with good methodological practice (< 25% complying studies) was found regarding behavioural analyses, discussion of model assumptions and uncertainties, modelling of value functions, and dealing with judgment inconsistencies. The five most reported challenges related to evidence and data synthesis; value system differences and participant selection issues; participant difficulties; methodological complexity and resource balance; and criteria and attributes modelling. A critical discussion on ways to address these challenges ensues.

Discussion: Results highlight the need for advancement in robust methodologies, procedures and tools to improve methodological quality of MCDA in HTA studies. Research pathways include developing new model features, good practice guidelines, technologies to enable participation and behavioural research.

Keywords: Health technology assessment; MCDA modelling; Methodological challenges; Methodological quality; Multi-criteria decision analysis; Systematic review.

Fig. 1

PRISMA flowchart describing study selection

Fig. 2

a Number of article publications over time; b number of publications by type; c number of publications according to health technology focus. Source: the authors from the literature

Fig. 3

Percentage of studies fully or at least partly following good practice considerations in the PROACTIVE-S sub-steps. Note: Abbreviations in use are defined in the last column of Table 1. Source: the authors from the literature

Fig. 4

Interconnectedness between the MCDA modelling steps with higher deviations from good methodological practice (on top), and the eight most reported limitations and challenges reported in MCDA in HTA studies (on bottom). Lines in the middle depict interrelations between those deviations and limitations/challenges, with topics near the lines (in capital letters) synthesising areas relevant for developing the state of the art within MCDA for HTA (topics discussed along the “Discussion” section). Source: the authors from the literature