Agent-based modeling of the COVID-19 pandemic in Florida

Abstract

The onset of the COVID-19 pandemic drove a widespread, often uncoordinated effort by research groups to develop mathematical models of SARS-CoV-2 to study its spread and inform control efforts. The urgent demand for insight at the outset of the pandemic meant early models were typically either simple or repurposed from existing research agendas. Our group predominantly uses agent-based models (ABMs) to study fine-scale intervention scenarios. These high-resolution models are large, complex, require extensive empirical data, and are often more detailed than strictly necessary for answering qualitative questions like "Should we lockdown?" During the early stages of an extraordinary infectious disease crisis, particularly before clear empirical evidence is available, simpler models are more appropriate. As more detailed empirical evidence becomes available, however, and policy decisions become more nuanced and complex, fine-scale approaches like ours become more useful. In this manuscript, we discuss how our group navigated this transition as we modeled the pandemic. The role of modelers often included nearly real-time analysis, and the massive undertaking of adapting our tools quickly. We were often playing catch up with a firehose of evidence, while simultaneously struggling to do both academic research and real-time decision support, under conditions conducive to neither. By reflecting on our experiences of responding to the pandemic and what we learned from these challenges, we can better prepare for future demands.

Keywords: Agent-based modeling; COVID-19; Decision support; Pandemic response.

Fig. 1

Major events in our COVID-19 modeling. The timeline (bottom) displays significant milestones of our lab’s modeling work, ABM development, and collaborations relative to the time series of COVID-19 reported cases in the state of Florida (top).

Fig. 2

Modeled disease state progression. All individuals begin susceptible ($S$), and upon successful exposure ($E$), will progress to an infectious ($I$) state. Infections can either be asymptomatic ($I_A$) or progress into mild ($I_M$), severe ($I_S$), or critical ($I_C$) symptomatic states. Only critical infections can result in death ($D$); others ultimately lead to recovery ($R$). Recovered individuals have partial immunity against infection that wanes over time, as well as partial protection against future disease that does not wane.

Fig. 3

Workplaces by type. Categorization of workplaces (roughly 2.3M total) in our Florida synthetic population in three dimensions: essential versus non-essential (blue), patronizable versus non-patronizable (red), and normal versus high risk (green). The matrix indicates sets of intersections of these categories and the bar chart above indicates the number of workplaces in that intersection. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

Map of synthetic population locations representing Ocala, FL. Modeled locations show realistic spatial distribution, with workplaces (blue dots) conforming to larger roads (bold black lines) while homes (green dots), schools (yellow-green boxes), long-term care facilities (brown boxes), and hospitals (red boxes) cluster around smaller roads (thin black lines). The ABM does not represent roads per se; they are added here for illustrative purposes only. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 5

Probability of detection over time. This figure shows trends in detection probability by peak disease severity, for the model at the point of the SMH work (top panel) and our subsequent assumptions (bottom panel). At both stages, we assumed that all infections resulting in death were detected. During the SMH work, we had assumed stable testing patterns leading up to and through the omicron wave. Shortly after making those projections, we assess that detection of mild disease likely dropped, based on Florida testing data. We later found additional evidence indicating a drop in detection when we began fitting to seroprevalence data.

Fig. 6

Reported vs. excess COVID-19 deaths in the state of Florida. Though some inconsistencies exist between these two data streams, we believe that it is a reasonable assumption that all excess deaths in Florida during the simulated time-period can be attributed to COVID-19 deaths (CDC, 2020, Excess deaths, 2023).

Fig. 7

Example of our in-house model dashboard. During model development and calibration, we used this visualization tool to easily compare model metrics (plotted in color) to relevant empirical data (plotted in black) or to otherwise capture trends for consideration (e.g. $R_t$ estimate, variant prevalence). This example plot shows a single run’s outputs compared to several different empirical data sources; notably during the SMH work, we were not yet considering seroprevalance and had not yet switched from reported COVID deaths to excess deaths. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 8

Representation of our automated SRP fitting procedure. This figure illustrates an in-process fitting attempt for a target anchor (solid orange circle). The eight-week fitting window (shaded regions) is divided into three distinct parts: (I in yellow) the two week period prior to the target anchor point, (II in light blue–green) the two week after the target, and (III in light purple) the four subsequent weeks. Region I accounts for the linear interpolation from the previous anchor to the current target. The periods that follow the target point emphasize primarily short term outcomes (Region II) to fit the current anchor, but also incorporate further horizon (Region III) outcomes to ensure later anchors will be reasonably fittable. Panel details: (A) All previously fit (solid gray dots), current (solid orange dot), and future (open orange dot) anchors points. Daily SRP values are linearly interpolated between anchor points (gray and orange lines). (B) Empirical (black) and simulated (blue) cumulative reported cases for the simulated period. For clarity, we zoom into the SRP (D) and cumulative case curves (E) around the current fitting window. For each day, the difference between simulated and empirical cumulative cases is calculated (i.e., error) (F). A weighting function (G) is multiplied by the calculated error to generated a final weighted error curve (H). The weighting favors the day of the anchor being fit and the two weeks before (I) and after (II), but a discounted weight is also applied further out (III). To calculate the goodness of fit for the current anchor value, the weighted error is summed over the fitting window and normalized by a three-day average of empirical cumulative reported cases centered on the current anchor day. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 9

Comparison of UF model projections as submitted to SMH Round 11 and after further development. Our submissions to SMH Round 11 (four scenarios exploring omicron assumptions; red and green curves) generally predicted an omicron wave that was too late, with too few reported cases for the number of hospitalizations and deaths. Our ultimate model (2023 version shown here; blue curves) recapitulated both the timing and height of the peaks using some parameters that were in the range of those considered in the SMH scenarios. Importantly, however, we adopted a waning immunity model that allowed for more reinfections, and a more aggressive external introduction model. We also stopped calibrating to reported deaths (open circles), which we found to be unreliable, and switched to excess deaths (closed circles). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)