How does policy modelling work in practice? A global analysis on the use of epidemiological modelling in health crises

Abstract

This study examines the use and translation of epidemiological modelling by policy and decision makers in response to the COVID-19 outbreak. Prior to COVID-19, there was little readiness for global health systems, and many science-policy networks were assembled ad-hoc. Moreover, in the field of epidemiological modelling, one with significant sudden influence, there is still no international guidance or standard of practice on how modelled evidence should guide policy during major health crises. Here we use a multi-country case study on the use of epidemiological modelling in emergency COVID-19 response, to examine the effective integration of crisis science and policy in different countries. We investigated COVID-19 modelling-policy systems and practices in 13 countries, spanning all six UN geographic regions. Data collection took the form of expert interviews with a range of national policy/ decision makers, scientific advisors, and modellers. We examined the current use of epidemiological modelling, introduced a classification framework for outbreak modelling and policy on which best practice can be structured, and provided preliminary recommendations for future practice. Full analysis and interpretation of the breadth of interview responses is presented, providing evidence for the current and future use of modelling in disease outbreaks. We found that interviewees in countries with a similar size and type of modelling infrastructure, and similar level of government interaction with modelling reported similar experiences and recommendations on using modelling in outbreak response. From this, we introduced a helpful grouping of country experience upon which a tailored future best practice could be structured. We concluded the article by outlining context-specific activities that modellers and policy actors could consider implementing in their own countries. This article serves as a first evidence base for the current use of modelling in a recent major health crisis and provides a robust framework for developing epidemiological modelling-to-policy best practice.

Fig 1. Infographic depicting seven broad challenges of pandemic modelling for policy, around the central equation of epidemic spread.

Reproduced from Hadley et al., 2021 [15].

Fig 2. The four types of national modelling infrastructure observed in our study.

One modelling team (1); multiple small teams functioning as one (no capacity to reproduce) (2); multiple teams and multiple models (capacity to reproduce) led by a committee of modellers (3); modellers and/or modelling teams do not collaborate, work in isolation, and feed results directly into government to be combined by a committee of non-modellers (4). Pictured: Individuals with hats represent modelling principal investigators/ senior modellers. Computers represent models. Boxes represent a modelling committee. For example, in South Africa (type 2), there was insufficient capacity to examine the same modelling questions with multiple models so teams divided the research questions among themselves and functioned as one team. In South Korea (type 4), modelling effort was disjointed - modelling teams did not collaborate or discuss findings and instead sent reports directly to the relevant policy actor.

Fig 3. Classification framework for outbreak modelling and policy.

Five categories of modelling-policy systems were identified and organised by similar experiences and recommendations on the use of outbreak modelling in policy. Type of modelling infrastructure and level of government interaction with modelling were identified as important contextual drivers when grouping country settings. The five categories of modelling-policy systems identified are: (A) Countries with small modelling capacity and likely high government linkage, such as Hong Kong; (B) Countries with large modelling capacity and high or very high government linkage, such as Kenya; (C) Countries where modelling teams worked in isolation, combined by non-modellers in government, such as South Korea; (D) Countries with a large modelling capacity with a primary government modelling team, such as Canada; (E) Countries with one government modelling team, such as Peru. Countries were placed in the same category if interviewees demonstrated similar beliefs and experiences on the use of outbreak modelling in policy. Note: For other pathogens and other health crises, each category may contain different countries.

Fig 4. Essentials for successful outbreak modelling and policy - data, systems, communication, and relationships.

This figure follows the style of earlier work Hadley et al, 2021.