Optimal adaptive nonpharmaceutical interventions to mitigate the outbreak of respiratory infections following the COVID-19 pandemic: a deep reinforcement learning study in Hong Kong, China

Abstract

Background: Long-lasting nonpharmaceutical interventions (NPIs) suppressed the infection of COVID-19 but came at a substantial economic cost and the elevated risk of the outbreak of respiratory infectious diseases (RIDs) following the pandemic. Policymakers need data-driven evidence to guide the relaxation with adaptive NPIs that consider the risk of both COVID-19 and other RIDs outbreaks, as well as the available healthcare resources.

Methods: Combining the COVID-19 data of the sixth wave in Hong Kong between May 31, 2022 and August 28, 2022, 6-year epidemic data of other RIDs (2014-2019), and the healthcare resources data, we constructed compartment models to predict the epidemic curves of RIDs after the COVID-19-targeted NPIs. A deep reinforcement learning (DRL) model was developed to learn the optimal adaptive NPIs strategies to mitigate the outbreak of RIDs after COVID-19-targeted NPIs are lifted with minimal health and economic cost. The performance was validated by simulations of 1000 days starting August 29, 2022. We also extended the model to Beijing context.

Findings: Without any NPIs, Hong Kong experienced a major COVID-19 resurgence far exceeding the hospital bed capacity. Simulation results showed that the proposed DRL-based adaptive NPIs successfully suppressed the outbreak of COVID-19 and other RIDs to lower than capacity. DRL carefully controlled the epidemic curve to be close to the full capacity so that herd immunity can be reached in a relatively short period with minimal cost. DRL derived more stringent adaptive NPIs in Beijing.

Interpretation: DRL is a feasible method to identify the optimal adaptive NPIs that lead to minimal health and economic cost by facilitating gradual herd immunity of COVID-19 and mitigating the other RIDs outbreaks without overwhelming the hospitals. The insights can be extended to other countries/regions.

Keywords: Covid-19; artificial intelligence; infectious diseases; machine learning; mathematical modelling; reinforcement learning.

Figure 1.

The framework of the SEVQICR model. S, E, V, I, IC, Q, R, and D represent susceptible, exposed, vaccinated, infected, infected and traced as contacts, quarantined, recovered, and dead, respectively.

Figure 2.

The demand for hospital beds without NPIs (Hong Kong).

Figure 3.

The epidemic curves of RIDs and the corresponding NPIs derived by DRL (Hong Kong). Level 1 (closure of entertainment place), Level 2 (school closure, work from home, and no dine-in after 9 pm), Level 3 (no dine-in after 6 pm), Level 4 (maximum number of people gathering is 2), and Level 5 (regional lockdown).

Figure 4.

The epidemic curves of RIDs and the corresponding NPIs derived by the threshold-based method with a threshold of 220 (Hong Kong). Level 1 (closure of entertainment place), Level 2 (school closure, work from home and no dine-in after 9 pm), Level 3 (no dine-in after 6 pm), Level 4 (maximum number of people gathering is 2), and Level 5 (regional lockdown).

Figure 5.

The epidemic curves of RIDs and the corresponding NPIs derived by RL (Hong Kong). Level 1 (closure of entertainment place), Level 2 (school closure, work from home, and no dine-in after 9 pm), Level 3 (no dine-in after 6 pm), Level 4 (maximum number of people gathering is 2), and Level 5 (regional lockdown).

Figure 6.

The demand for hospital beds without NPIs (Beijing).

Figure 7.

The epidemic curves of RIDS and the corresponding NPIs derived by DRL (Beijing without temporary mobile field hospitals). Level 1 (closure of entertainment place), Level 2 (school closure, work from home, and no dine-in after 9 pm), Level 3 (no dine-in after 6 pm), Level 4 (maximum number of people gathering is 2), and Level 5 (regional lockdown).

Figure 8.

The epidemic curves of RIDS and the corresponding NPIs derived by DRL (Beijing with temporary mobile field hospitals). Level 1 (closure of entertainment place), Level 2 (school closure, work from home, and no dine-in after 9 pm), Level 3 (no dine-in after 6 pm), Level 4 (maximum number of people gathering is 2), and Level 5 (regional lockdown).