Decision-analytic models: current methodological challenges

Abstract

Modelers seeking to help inform decisions about insurance (public or private) coverage of the cost of pharmaceuticals or other health care interventions face various methodological challenges. In this review, which is not meant to be comprehensive, we cover those that in our experience are most vexing. The biggest challenge is getting decision makers to trust the model. This is a major problem because most models undergo only cursory validation; our field has lacked the motivation, time, and data to properly validate models intended to inform health care decisions. Without documented, adequate validation, there is little basis for decision makers to have confidence that the model's results are credible and should be used in a health technology appraisal. A fundamental problem for validation is that the models are very artificial and lack sufficient depth to adequately represent the reality they are simulating. Typically, modelers assume that all resources have infinite capacity so any patient needing care receives it immediately; there are no waiting times or queues, contrary to the common experience in actual practice. Moreover, all the patients enter the model simultaneously at time zero rather than over time as happens in actuality; differences between patients are ignored or minimized and structural modeling choices that make little sense (e.g., using states to represent events) are forced by commitment to a technique (and even to specific spreadsheet software!). The resulting structural uncertainty is rarely addressed, because methods are lacking and even probabilistic analysis of parameter uncertainty suffers from weak consideration of correlation and arbitrary distribution choices. Stakeholders must see to it that models are fit for the stated purpose and provide the best possible estimates given available data-the decisions at stake deserve nothing less.