Real-World Cost-Effectiveness Analysis: How Much Uncertainty Is in the Results?

Abstract

Cost-effectiveness analyses of new cancer treatments in real-world settings (e.g., post-clinical trials) inform healthcare decision makers about their healthcare investments for patient populations. The results of these analyses are often, though not always, presented with statistical uncertainty. This paper identifies five ways to characterize statistical uncertainty: (1) a 95% confidence interval (CI) for the incremental cost-effectiveness ratio (ICER); (2) a 95% CI for the incremental net benefit (INB); (3) an INB by willingness-to-pay (WTP) plot; (4) a cost-effectiveness acceptability curve (CEAC); and (5) a cost-effectiveness scatterplot. It also explores their usage in 22 articles previously identified by a rapid review of real-world cost effectiveness of novel cancer treatments. Seventy-seven percent of these articles presented uncertainty results. The majority those papers (59%) used administrative data to inform their analyses while the remaining were conducted using models. Cost-effectiveness scatterplots were the most commonly used method (34.3%), with 40% indicating high levels of statistical uncertainty, suggesting the possibility of a qualitatively different result from the estimate given. Understanding the necessity for and the meaning of uncertainty in real-world cost-effectiveness analysis will strengthen knowledge translation efforts to improve patient outcomes in an efficient manner.

Keywords: cancer; cancer interventions; cost effectiveness; economic evaluation; healthcare; real-world interventions; statistics; uncertainty.

Figure 1

(A) Incremental cost-effectiveness ratio (ICER) illustrated on a cost-effectiveness plane. (B) The ICER and its 95% confidence interval illustrated on a cost-effectiveness plane. (C) The scatterplot approach to showing ICER uncertainty on a cost-effectiveness plane.

Figure 1

(A) Incremental cost-effectiveness ratio (ICER) illustrated on a cost-effectiveness plane. (B) The ICER and its 95% confidence interval illustrated on a cost-effectiveness plane. (C) The scatterplot approach to showing ICER uncertainty on a cost-effectiveness plane.

Figure 2

The cost-effectiveness acceptability curve (CEAC).

Figure 3

Incremental net benefit by willingness to pay.

Figure 4

Quality of statistical uncertainty techniques used by analysis type.

Figure 5

Radar graph illustrating the quantity of uncertainty methods by paper [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34].

Figure 6

Quantity of statistical uncertainty techniques by year.

Figure 7

Location of uncertainty findings in scatterplots. Note: Location and meaning of uncertainty findings in scatterplots. The horizontal axis indicates location of the uncertainty in a cost-effectiveness plane and the vertical axis indicates the interpretation of the uncertainty. The numbers to the left of the “bars” are the row percentages across all (both dataset- and model-based analyses) papers. The quadrant interpretations are as follows: North-East (NE): higher cost, higher effect; NE, North-West (NW): higher cost, uncertain effect; NE, South-East (SE): higher effect, uncertain cost; NW, South-West (SW): lower effect, uncertain cost; NE, SE, NW: uncertain cost, uncertain effect; NE, SE, NW, SW: uncertain cost, uncertain effect.