Ensemble forecast modeling for the design of COVID-19 vaccine efficacy trials

Abstract

To rapidly evaluate the safety and efficacy of COVID-19 vaccine candidates, prioritizing vaccine trial sites in areas with high expected disease incidence can speed endpoint accrual and shorten trial duration. Mathematical and statistical forecast models can inform the process of site selection, integrating available data sources and facilitating comparisons across locations. We recommend the use of ensemble forecast modeling - combining projections from independent modeling groups - to guide investigators identifying suitable sites for COVID-19 vaccine efficacy trials. We describe an appropriate structure for this process, including minimum requirements, suggested output, and a user-friendly tool for displaying results. Importantly, we advise that this process be repeated regularly throughout the trial, to inform decisions about enrolling new participants at existing sites with waning incidence versus adding entirely new sites. These types of data-driven models can support the implementation of flexible efficacy trials tailored to the outbreak setting.

Keywords: Efficacy trial; Ensemble modeling; Forecast model; Trial planning.

Fig. 1

Hypothetical model stacking procedure for a target trial site. The procedure integrates projections from three independent models. For each model, cumulative incidence during a target period is projected and then summarized in bins of 1% width (left). Models are equally weighted and then stacked in an ensemble model projection (right). This process is repeated for each site. Figure modeled after Ray and Reich (2018).

Fig. 2

Screenshot from an interactive tool to display ensemble model projections and associated summary statistics. The rows are sortable and can be selected or deselected to form a hypothetical trial. Additional columns can be added to describe site features that are useful to investigators.