Prioritizing investments in rapid response vaccine technologies for emerging infections: A portfolio decision analysis

Abstract

This study reports on the application of a Portfolio Decision Analysis (PDA) to support investment decisions of a non-profit funder of vaccine technology platform development for rapid response to emerging infections. A value framework was constructed via document reviews and stakeholder consultations. Probability of Success (PoS) data was obtained for 16 platform projects through expert assessments and stakeholder portfolio preferences via a Discrete Choice Experiment (DCE). The structure of preferences and the uncertainties in project PoS suggested a non-linear, stochastic value maximization problem. A simulation-optimization algorithm was employed, identifying optimal portfolios under different budget constraints. Stochastic dominance of the optimization solution was tested via mean-variance and mean-Gini statistics, and its robustness via rank probability analysis in a Monte Carlo simulation. Project PoS estimates were low and substantially overlapping. The DCE identified decreasing rates of return to investing in single platform types. Optimal portfolio solutions reflected this non-linearity of platform preferences along an efficiency frontier and diverged from a model simply ranking projects by PoS-to-Cost, despite significant revisions to project PoS estimates during the review process in relation to the conduct of the DCE. Large confidence intervals associated with optimization solutions suggested significant uncertainty in portfolio valuations. Mean-variance and Mean-Gini tests suggested optimal portfolios with higher expected values were also accompanied by higher risks of not achieving those values despite stochastic dominance of the optimal portfolio solution under the decision maker's budget constraint. This portfolio was also the highest ranked portfolio in the simulation; though having only a 54% probability of being preferred to the second-ranked portfolio. The analysis illustrates how optimization modelling can help health R&D decision makers identify optimal portfolios in the face of significant decision uncertainty involving portfolio trade-offs. However, in light of such extreme uncertainty, further due diligence and ongoing updating of performance is needed on highly risky projects as well as data on decision makers' portfolio risk attitude before PDA can conclude about optimal and robust solutions.

Fig 1. Example choice set in the DCE.

Fig 2

a, b. Project PoS (Mean, 95% CI). Displaying the mean and variance in PoS of projects generated by the simulation (10,000 iterations) under methods steps 3.1 and 3.2 (initial versus final reviewer assessments). c. Project PoS distributions (final reviewer assessments). Displaying the final project PoS distributions for the 16 projects assessed. Each bar chart represents another project, with the vertical axis indicating the frequency of occurrence of PoS estimates out of 10,000 simulation iterations, and the horizontal axis indicating different levels of PoS estimates emerging across the 10,000 simulation iterations.

Fig 3

a. Portfolio value associated with probability of ≥1 project successfully developed per platform type (initial reviewer assessments). Mean POS_≥1(k) and V_p estimates are calculated by running the optimization process under step 5 separately for each platform type k, as follows: maximizing V_p several times, each time incrementally increasing the number of projects (decision variables in the model) entering the portfolio, and repeating this process until all projects are added. b. Portfolio value associated with probability of ≥1 project successfully developed per platform type (final reviewer assessments). Mean POS_≥1(k) and V_p estimates are calculated by running the optimization process under step 5 separately for each platform type k, as follows: maximizing V_p several times, each time incrementally increasing the number of projects (decision variables in the model) entering the portfolio, and repeating this process until all projects are added.

Fig 4

a. Optimal Frontier by maximizing portfolio value drawing from final reviewer assessments of project PoS. Fig 4A shows the efficiency frontier constructed by the optimization process under step 5, drawing from final reviewer assessments of project PoS. This is compared against the frontier that would have been generated if projects were simply ranked by expected PoS-to-Cost, then incrementally added to the portfolio without accounting for whether the resulting portfolios would maximize V_p under different budget constraints. b. Optimal Frontier by maximizing portfolio value drawing from initial reviewer assessments of project PoS. Fig 4B shows the efficiency frontier constructed by the optimization process under step 5, drawing from initial reviewer assessments of project PoS. This is compared against the frontier that would have been generated if projects were simply ranked by expected PoS-to-Cost, then incrementally added to the portfolio without accounting for whether the resulting portfolios would maximize V_p under different budget constraints.

Fig 5. Optimal frontiers by mean-variance, mean-semivariance, mean-standard deviation, and mean-absolute deviation, under a US$140 million constraint.

Fig 6. Optimal Frontier by Mean-Gini performance of the portfolio, under a US$140 million constraint.

Fig 7

a. Probability ranges of optimal portfolio outranking alternative portfolios under a US$ 140 million constraint. b. Project composition of portfolio alternatives the optimal portfolio outranks under a US$ 140 million constraint.