Bayesian emulation and history matching of JUNE

Abstract

We analyze JUNE: a detailed model of COVID-19 transmission with high spatial and demographic resolution, developed as part of the RAMP initiative. JUNE requires substantial computational resources to evaluate, making model calibration and general uncertainty analysis extremely challenging. We describe and employ the uncertainty quantification approaches of Bayes linear emulation and history matching to mimic JUNE and to perform a global parameter search, hence identifying regions of parameter space that produce acceptable matches to observed data, and demonstrating the capability of such methods. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Keywords: Bayes linear; calibration; disease models; emulation; history matching.

Figure 1.

An emulator of a one-dimensional toy model, where $f(x)=\sin (2\pi (x-0.1)/0.4)$, for the first wave/iteration, using just six runs (left panel), and for the second wave, using two additional runs (right panel). The emulator’s expectation ${\mathrm{E}}_D[f(x)]$ and credible intervals ${\mathrm{E}}_D[f(x)]\pm 3\sqrt{{\mathrm{Var}}_{D_i}(f_i(x))}$ are given by the blue and red lines, respectively, with the observed data $z$ that we wish to match to as the black horizontal line (with errors). The implausibility $I(x)$ is represented by the coloured bar along the $x$-axis, with dark blue implying $I(x)>3$, light blue $2.5<I(x)<3$ and yellow ($I(x)<1$). (Online version in colour.)

Figure 2.

Daily deaths in hospital wards and ICU in 2020, by region. The smoothed version used in the HM is also shown. (Online version in colour.)

Figure 3.

Estimates for the coefficients $b_{ij}$ of the linear terms $g_{ij}(x_{A_i})=x_{A_i}^j$ that are found to feature in the emulators for total deaths in England for the first iteration/wave of runs, where $i$ labels the time point ($x$-axis) and $j$ labels the inputs ($y$-axis). Red/blue represents positive/negative dependencies of $f_i(x)$ on that input, respectively, standardized as proportions of the largest coefficient for that time point. A finer temporal resolution is used here for added clarity. Note that this plot shows the time-dependent sensitivity of the model to the inputs, but that the actual inputs $x$ do not vary over time. (Online version in colour.)

Figure 4.

The JUNE output for total daily deaths in England in 2020, for several iterations of the HM process. The smoothed and noisy data, along with the combined uncertainties due to ${\sigma }_e$ and ${\sigma }_{ϵ}$, are shown in black. (Online version in colour.)

Figure 5.

The optical depth $\rho (x^{\prime })$ of various two-dimensional projections of the full 20-dimensional non-implausible region ${\mathcal{X}}_5$ found after the 5th iteration. The 12 most constrained inputs are show, labelled on the diagonal (the remaining eight inputs were relatively unconstrained). The colour scales are standardized and linear in depth, with yellow showing maximum depth for that projection and purple/black showing minimum/zero depth. This region corresponds to the red runs in figure 4. (Online version in colour.)

Figure 6.

A single JUNE run (red lines), from the second exploratory iteration (i.e. one of the blue lines in figure 4). The panels show hospital deaths (rows 1 and 2, viewed in landscape) and total deaths (rows 3 and 4, viewed in landscape) for England and the seven regions, as given in the plot titles. The black points give the (unsmoothed) death data, and the combined uncertainties due to ${\sigma }_e$ and ${\sigma }_{ϵ}$ are shown as the blue lines. (Online version in colour.)