Comparison between simulated scenarios and Swedish COVID-19 cases throughout the pandemic

Abstract

This study assesses the accuracy of COVID-19 scenarios for new infections produced by the Swedish Public Health Agency (PHAS) from December 1, 2020, to March 20, 2023. We introduce a Similarity Error ([Formula: see text]), which evaluates the dissimilarity between simulated and observed case time series using the following attributes: area under the curves, peak timings, and growth/decline rates before and after peaks. Rather than using an arbitrary cut-off, we used a threshold determined through Receiver Operating Characteristic (ROC) analysis, with performance evaluated using the Area Under the Curve (AUC), based on true positives identified by visual inspection for categorization. To further evaluate [Formula: see text]'s effectiveness, we conducted a sensitivity analysis across the full range of possible threshold values within the unit interval. Applying [Formula: see text] with an optimal threshold determined through ROC-analysis 7 rounds out of 11 rounds were classified as having one or more similar scenarios, including the 6 rounds identified by visual inspection. Our findings indicate that, despite the challenges of a rapidly evolving epidemic, PHAS delivered simulations that reflected real-world trends in most of the rounds.

Keywords: COVID-19; Scenario analysis; Simulation similarity; Time series comparison.

Fig. 1

PHAS released thirteen rounds of simulations of new infections, from December 1, 2020, to March 20, 2023, with each round featuring one to three scenarios. (a) Each round is represented by a unique colour. The figure displays the raw daily national case counts (in green) and the smoothed 7-day rolling averages (SDCC in blue). Intervals for simulation rounds are highlighted as bars, publication dates are marked with downward arrows, and shaded areas indicate periods dominated by different variants. (b) The table provides details for each round, including the number of scenarios, publication date, reporting scale, period covered, duration in days, and the extent of overlap between the simulation period and the dominance of various variants.

Fig. 2

The smoothed 7-day rolling averages daily national (Riket) case count (dashed blue line) across different simulation rounds, along with the simulated case numbers for each specific scenario.

Fig. 3

Similarity classification of the national (Riket) simulation rounds. (a) The four error measures classified based on thresholds determined through ROC analysis. The specific thresholds are as follows: , , , . (b) Upset figure displaying how many distinct rounds each error measures or combination of error measures exhibited similarity simultaneously.

Fig. 4

Classification performance of . a) Over a range of values where each of the national (Riket) scenarios per round are classified and colour coded accordingly. b) Number of rounds and c) Number of scenarios classified as Similar over the range of values.

Fig. 5

The categorization of all regions and scenarios for all simulations rounds and considered regions. Based on the configuration of determined by the national ROC analysis (Riket, representing the national level). Respective -value is displayed in each tile and subsequently colour coded according to classification.