A Methodological Framework for Economic Evaluation of Operational Response to Vector-Borne Diseases Based on Early Warning Systems

Abstract

Despite significant advances in improving the predictive models for vector-borne diseases, only a few countries have integrated an early warning system (EWS) with predictive and response capabilities into their disease surveillance systems. The limited understanding of forecast performance and uncertainties by decision-makers is one of the primary factors that precludes its operationalization in preparedness and response planning. Further, predictive models exhibit a decrease in forecast skill with longer lead times, a trade-off between forecast accuracy and timeliness and effectiveness of action. This study presents a methodological framework to evaluate the economic value of EWS-triggered responses from the health system perspective. Assuming an operational EWS in place, the framework makes explicit the trade-offs between forecast accuracy, timeliness of action, effectiveness of response, and costs, and uses the net benefit analysis, which measures the benefits of taking action minus the associated costs. Uncertainty in disease forecasts and other parameters is accounted for through probabilistic sensitivity analysis. The output is the probability distribution of the net benefit estimates at given forecast lead times. A non-negative net benefit and the probability of yielding such are considered a general signal that the EWS-triggered response at a given lead time is economically viable. In summary, the proposed framework translates uncertainties associated with disease forecasts and other parameters into decision uncertainty by quantifying the economic risk associated with operational response to vector-borne disease events of potential importance predicted by an EWS. The goal is to facilitate a more informed and transparent public health decision-making under uncertainty.

Figure 1.

An illustration of the relationship between forecast lead time (i) in weeks or months and forecast accuracy as measured by mean-square error (MSE), assuming a maximum forecast lead time of n. The uncertainty at different lead times could also be reported using probabilistic estimates, and one of the most used measures is credible intervals.

Figure 2.

(A) An illustration of the relationship between forecast lead time (i) in weeks or months and response effectiveness (e_i) and its associated uncertainty given a certain level of budget and (B) cost of response activities to achieve a certain effectiveness level, assuming a maximum lead time of n weeks or months.

Figure 3.

Hypothetical illustration of outputs from an operational early warning system. (A) Disease forecast output at time point t. (B) Disease forecast output at time point t for an intervention of effectiveness e_i deployed at lead time i.

Figure 4.

The hypothetical probability distribution function (PDF) of NB_t(i) for a specific forecast lead time i.