Field testing of a multicriteria decision analysis (MCDA) framework for coverage of a screening test for cervical cancer in South Africa

Abstract

Background: Systematic and transparent approaches to priority setting are needed, particularly in low-resource settings, to produce decisions that are sound and acceptable to stakeholders. The EVIDEM framework brings together Health Technology Assessment (HTA) and multi-criteria decision analysis (MCDA) by proposing a comprehensive set of decision criteria together with standardized processes to support decisionmaking. The objective of the study was to field test the framework for decisionmaking on a screening test by a private health plan in South Africa.

Methods: Liquid-based cytology (LBC) for cervical cancer screening was selected by the health plan for this field test. An HTA report structured by decision criterion (14 criteria organized in the MCDA matrix and 4 contextual criteria) was produced based on a literature review and input from the health plan. During workshop sessions, committee members 1) weighted each MCDA decision criterion to express their individual perspectives, and 2) to appraise LBC, assigned scores to each MCDA criterion on the basis of the by-criterion HTA report.Committee members then considered the potential impacts of four contextual criteria on the use of LBC in the context of their health plan. Feedback on the framework and process was collected through discussion and from a questionnaire.

Results: For 9 of the MCDA matrix decision criteria, 89% or more of committee members thought they should always be considered in decisionmaking. Greatest weights were given to the criteria "Budget impact", "Cost-effectiveness" and "Completeness and consistency of reporting evidence". When appraising LBC for cervical cancer screening, the committee assigned the highest scores to "Relevance and validity of evidence" and "Disease severity". Combination of weights and scores yielded a mean MCDA value estimate of 46% (SD 7%) of the potential maximum value. Overall, the committee felt the framework brought greater clarity to the decisionmaking process and was easily adaptable to different types of health interventions.

Conclusions: The EVIDEM framework was easily adapted to evaluating a screening technology in South Africa, thereby broadening its applicability in healthcare decision making.

Figure 1

Study plan.

Figure 2

Mean weights assigned to each decision criterion of the MCDA matrix by committee members. A five point weighting scale was used with 1 for lowest weight and 5 for highest weight.

Figure 3

Mean scores assigned to each decision criterion of the MCDA matrix by committee members for the appraisal of liquid based cytology. A four point scoring scale was used with 0 for lowest score and 3 for highest score.

Figure 4

MCDA value estimate of liquid based cytology for cervical cancer screening based on weights and scores assigned by committee members. Weights were normalized across the 14 criteria and scores are presented on a scale of 0 to 1. *MCDA estimate was obtained using a linear model combining normalized weights and scores for each decision criteria. For an intervention to achieve close to 1 on this scale, it would have to cure an endemic disease, demonstrate major improvement in safety, efficacy, and patient reported outcomes compared to limited existing approaches and result in major healthcare savings.