Review

. 2023 Jul;32(7):1603-1625.

doi: 10.1002/hec.4681. Epub 2023 Apr 20.

Guidance on the use of complex systems models for economic evaluations of public health interventions

Penny R Breeze¹, Hazel Squires¹, Kate Ennis², Petra Meier³, Kate Hayes¹, Nik Lomax⁴, Alan Shiell⁵, Frank Kee⁶, Frank de Vocht^{7

8}, Martin O'Flaherty⁹, Nigel Gilbert¹⁰, Robin Purshouse¹¹, Stewart Robinson¹², Peter J Dodd¹, Mark Strong¹, Suzy Paisley¹³, Richard Smith¹⁴, Andrew Briggs¹⁵, Lion Shahab¹⁶, Jo-An Occhipinti¹⁷, Kenny Lawson¹⁷, Thomas Bayley¹⁸, Robert Smith¹, Jennifer Boyd^{1

19}, Visakan Kadirkamanathan¹¹, Richard Cookson²⁰, Monica Hernandez-Alava¹, Christopher H Jackson²¹, Amanda Karapici²², Franco Sassi²³, Peter Scarborough²⁴, Uwe Siebert^{25

26

27

28}, Eric Silverman¹⁹, Luke Vale²⁹, Cathal Walsh³⁰, Alan Brennan¹

Affiliations

¹ School of Health and Related Research, University of Sheffield, Sheffield, UK.
² British Medical Journal Technology Appraisal Group, London, UK.
³ MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Scotland, UK.
⁴ School of Geography, University of Leeds, Leeds, UK.
⁵ Department of Public Health, LaTrobe University, Melbourne, Australia.
⁶ Centre for Public Health, Queen's University Belfast, Belfast, UK.
⁷ Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.
⁸ NIHR Applied Research Collaboration West (ARC West), Bristol, UK.
⁹ Department of Public Health, Policy and Systems, University of Liverpool, Liverpool, UK.
¹⁰ CRESS, University of Surrey, Guildford, UK.
¹¹ Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, UK.
¹² Business School, University of Newcastle, Newcastle, UK.
¹³ Lumanity-HEOR, South Yorkshire, Sheffield, UK.
¹⁴ College of Medicine and Health, University of Exeter, Exeter, UK.
¹⁵ London School of Hygiene & Tropical Medicine, London, UK.
¹⁶ Department of Behavioural Science and Health, UCL, London, UK.
¹⁷ Brain and Mind Centre, University of Sydney, New South Wales, Camperdown, Australia.
¹⁸ United Kingdom Health Security Agency, Birmingham, UK.
¹⁹ MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Glasgow, UK.
²⁰ Centre for Health Economics, University of York, Heslington, UK.
²¹ MRC Biostatistics Unit, University of Cambridge, Cambridge, UK.
²² NIHR SPHR, London School of Hygiene and Tropical Medicine, London, UK.
²³ Centre for Health Economics & Policy Innovation, Imperial College Business School, London, UK.
²⁴ Nuffield Department of Population Health, University of Oxford, Oxfordshire, Oxford, UK.
²⁵ Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL - University for Health Sciences and Technology, Hall in Tirol, Tyrol, Austria.
²⁶ Division of Health Technology Assessment and Bioinformatics, ONCOTYROL - Center for Personalized Cancer Medicine, Innsbruck, Austria.
²⁷ Center for Health Decision Science, Departments of Epidemiology and Health Policy & Management, Harvard T.H. Chan School of Public Health, Massachusetts, Boston, USA.
²⁸ Program on Cardiovascular Research, Institute for Technology Assessment and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Massachusetts, Boston, USA.
²⁹ Health Economics Group, Population Health Sciences Institute, Newcastle University, Newcastle, UK.
³⁰ Health Research Institute and MACSI, University of Limerick, Limerick, Ireland.

PMID: 37081811
PMCID: PMC10947434
DOI: 10.1002/hec.4681

Review

Guidance on the use of complex systems models for economic evaluations of public health interventions

Penny R Breeze et al. Health Econ. 2023 Jul.

. 2023 Jul;32(7):1603-1625.

doi: 10.1002/hec.4681. Epub 2023 Apr 20.

Authors

Affiliations

¹ School of Health and Related Research, University of Sheffield, Sheffield, UK.
² British Medical Journal Technology Appraisal Group, London, UK.
³ MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Scotland, UK.
⁴ School of Geography, University of Leeds, Leeds, UK.
⁵ Department of Public Health, LaTrobe University, Melbourne, Australia.
⁶ Centre for Public Health, Queen's University Belfast, Belfast, UK.
⁷ Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.
⁸ NIHR Applied Research Collaboration West (ARC West), Bristol, UK.
⁹ Department of Public Health, Policy and Systems, University of Liverpool, Liverpool, UK.
¹⁰ CRESS, University of Surrey, Guildford, UK.
¹¹ Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, UK.
¹² Business School, University of Newcastle, Newcastle, UK.
¹³ Lumanity-HEOR, South Yorkshire, Sheffield, UK.
¹⁴ College of Medicine and Health, University of Exeter, Exeter, UK.
¹⁵ London School of Hygiene & Tropical Medicine, London, UK.
¹⁶ Department of Behavioural Science and Health, UCL, London, UK.
¹⁷ Brain and Mind Centre, University of Sydney, New South Wales, Camperdown, Australia.
¹⁸ United Kingdom Health Security Agency, Birmingham, UK.
¹⁹ MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Glasgow, UK.
²⁰ Centre for Health Economics, University of York, Heslington, UK.
²¹ MRC Biostatistics Unit, University of Cambridge, Cambridge, UK.
²² NIHR SPHR, London School of Hygiene and Tropical Medicine, London, UK.
²³ Centre for Health Economics & Policy Innovation, Imperial College Business School, London, UK.
²⁴ Nuffield Department of Population Health, University of Oxford, Oxfordshire, Oxford, UK.
²⁵ Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL - University for Health Sciences and Technology, Hall in Tirol, Tyrol, Austria.
²⁶ Division of Health Technology Assessment and Bioinformatics, ONCOTYROL - Center for Personalized Cancer Medicine, Innsbruck, Austria.
²⁷ Center for Health Decision Science, Departments of Epidemiology and Health Policy & Management, Harvard T.H. Chan School of Public Health, Massachusetts, Boston, USA.
²⁸ Program on Cardiovascular Research, Institute for Technology Assessment and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Massachusetts, Boston, USA.
²⁹ Health Economics Group, Population Health Sciences Institute, Newcastle University, Newcastle, UK.
³⁰ Health Research Institute and MACSI, University of Limerick, Limerick, Ireland.

PMID: 37081811
PMCID: PMC10947434
DOI: 10.1002/hec.4681

Abstract

To help health economic modelers respond to demands for greater use of complex systems models in public health. To propose identifiable features of such models and support researchers to plan public health modeling projects using these models. A working group of experts in complex systems modeling and economic evaluation was brought together to develop and jointly write guidance for the use of complex systems models for health economic analysis. The content of workshops was informed by a scoping review. A public health complex systems model for economic evaluation is defined as a quantitative, dynamic, non-linear model that incorporates feedback and interactions among model elements, in order to capture emergent outcomes and estimate health, economic and potentially other consequences to inform public policies. The guidance covers: when complex systems modeling is needed; principles for designing a complex systems model; and how to choose an appropriate modeling technique. This paper provides a definition to identify and characterize complex systems models for economic evaluations and proposes guidance on key aspects of the process for health economics analysis. This document will support the development of complex systems models, with impact on public health systems policy and decision making.

Keywords: complex systems; economic modeling; public health.

PubMed Disclaimer

Conflict of interest statement

The authors have declared funding related to their participation in this research, and have completed conflict of interest disclosure forms.

Figures

**FIGURE 1**
A decision tool to identify whether a complex systems model is recommended to address a decision problem.

See this image and copyright information in PMC

Cited by

Artificial Intelligence to Automate Health Economic Modelling: A Case Study to Evaluate the Potential Application of Large Language Models.
Reason T, Rawlinson W, Langham J, Gimblett A, Malcolm B, Klijn S. Reason T, et al. Pharmacoecon Open. 2024 Mar;8(2):191-203. doi: 10.1007/s41669-024-00477-8. Epub 2024 Feb 10. Pharmacoecon Open. 2024. PMID: 38340276 Free PMC article.
Modeling the Impact of Vaping: What We Need to Know and Which Methods to Use.
Shahab L. Shahab L. Nicotine Tob Res. 2025 Feb 24;27(3):561-563. doi: 10.1093/ntr/ntae204. Nicotine Tob Res. 2025. PMID: 39223905 Free PMC article. No abstract available.
Integrated disease management, optimal effort allocation between prevention and control.
Bonneau M, Mandonnet N, Marie-Magdeleine C, Archimède H, Bambou JC. Bonneau M, et al. Sci Rep. 2025 Aug 18;15(1):30269. doi: 10.1038/s41598-025-16318-5. Sci Rep. 2025. PMID: 40826222 Free PMC article.
The Knowledge and Application of Economics in Healthcare in a High-Income Country Today: The Case of Belgium.
Standaert B, Vandenberghe D, Connolly MP, Hellings J. Standaert B, et al. J Mark Access Health Policy. 2024 Sep 4;12(3):264-279. doi: 10.3390/jmahp12030021. eCollection 2024 Sep. J Mark Access Health Policy. 2024. PMID: 39315121 Free PMC article.
Future Projections of Diabetes-Related Amputations in Eastern Saudi Arabia During 2022-2045 Using a Validated Epidemiological Model.
Al-Quwaidhi AJ, AlSaleh EM. Al-Quwaidhi AJ, et al. Cureus. 2023 Sep 26;15(9):e45972. doi: 10.7759/cureus.45972. eCollection 2023 Sep. Cureus. 2023. PMID: 37900427 Free PMC article.

See all "Cited by" articles

References

1. Ali, M. S. , Prieto‐Alhambra, D. , Lopes, L. C. , Ramos, D. , Bispo, N. , Ichihara, M. Y. , Pescarini, J. M. , Williamson, E. , Fiaccone, R. L. , Barreto, M. L. , & Smeeth, L. (2019). Propensity score methods in health technology assessment: Principles, extended applications, and recent advances. Frontiers in Pharmacology, 10, 973. 10.3389/fphar.2019.00973 - DOI - PMC - PubMed
1. Atkinson, J.‐A. , Page, A. , Wells, R. , Milat, A. , & Wilson, A. (2015). A modelling tool for policy analysis to support the design of efficient and effective policy responses for complex public health problems. Implementation Science, 10(1), 1–9. 10.1186/s13012-015-0221-5 - DOI - PMC - PubMed
1. Atkinson, J. A. , Skinner, A. , Hackney, S. , Mason, L. , Heffernan, M. , Currier, D. , King, K. , & Pirkis, J. (2020). Systems modelling and simulation to inform strategic decision making for suicide prevention in rural New South Wales (Australia). Australian and New Zealand Journal of Psychiatry, 54(9), 892–901. 10.1177/0004867420932639 - DOI - PubMed
1. Balci, O. (1994). Validation, verification, and testing techniques throughout the life cycle of a simulation study. In Proceedings of winter simulation conference (pp. 215–220). IEEE.
1. Basu, S. , & Andrews, J. (2013). Complexity in mathematical models of public health policies: A guide for consumers of models. PLoS Medicine, 10(10), e1001540. 10.1371/journal.pmed.1001540 - DOI - PMC - PubMed

Publication types

Actions
Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Guidance on the use of complex systems models for economic evaluations of public health interventions

Affiliations

Guidance on the use of complex systems models for economic evaluations of public health interventions

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

Publication types

MeSH terms

Related information

Grants and funding

LinkOut - more resources

Full Text Sources