Identifying best modelling practices for tobacco control policy simulations: a systematic review and a novel quality assessment framework

Abstract

Background: Policy simulation models (PSMs) have been used extensively to shape health policies before real-world implementation and evaluate post-implementation impact. This systematic review aimed to examine best practices, identify common pitfalls in tobacco control PSMs and propose a modelling quality assessment framework.

Methods: We searched five databases to identify eligible publications from July 2013 to August 2019. We additionally included papers from Feirman et al for studies before July 2013. Tobacco control PSMs that project tobacco use and tobacco-related outcomes from smoking policies were included. We extracted model inputs, structure and outputs data for models used in two or more included papers. Using our proposed quality assessment framework, we scored these models on population representativeness, policy effectiveness evidence, simulated smoking histories, included smoking-related diseases, exposure-outcome lag time, transparency, sensitivity analysis, validation and equity.

Findings: We found 146 eligible papers and 25 distinct models. Most models used population data from public or administrative registries, and all performed sensitivity analysis. However, smoking behaviour was commonly modelled into crude categories of smoking status. Eight models only presented overall changes in mortality rather than explicitly considering smoking-related diseases. Only four models reported impacts on health inequalities, and none offered the source code. Overall, the higher scored models achieved higher citation rates.

Conclusions: While fragments of good practices were widespread across the reviewed PSMs, only a few included a 'critical mass' of the good practices specified in our quality assessment framework. This framework might, therefore, potentially serve as a benchmark and support sharing of good modelling practices.

Keywords: public policy; smoking caused disease; socioeconomic status.

Figure 1

An adapted Preferred Reporting Items for Systematic Reviews and Meta-Analysis flow chart of identified studies.

Figure 2

Occurrence of model outcome types (some models included more than one output type).

Figure 3

Occurrence of number of disease groups simulated by models. Considered disease groups: cancers, chronic obstructive pulmonary disease, cardiovascular disease and any other smoking-related diseases. The models with no diseases explicitly modelled either calculated all-cause mortality directly from smoking status or used the number of smokers or non-smokers without explicitly modelling disease pathways.

Figure 4

Occurrence of types of model validations (some models used more than one validation type).