Replication of Published Health Economic Obesity Models: Assessment of Facilitators, Hurdles and Reproduction Success

Abstract

Objectives: This research aims to (1) replicate published health economic models, (2) compare reproduced results with original results, (3) identify facilitators and hurdles to model replicability and determine reproduction success, and (4) suggest model replication reporting standards to enhance model reproducibility, in the context of health economic obesity models.

Methods: Four health economic obesity models simulating an adult UK population were identified, selected for replication, and evaluated using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Reproduction results were compared to original results, focusing on cost-effectiveness outcomes, and the resulting reproduction success was assessed by published criteria. Replication facilitators and hurdles were identified and transferred into related reporting standards.

Results: All four case studies were state-transition models simulating costs and quality-adjusted life-years (QALYs). Comparing original versus reproduction outcomes, the following deviation ranges were observed: costs - 3.9 to 16.1% (mean over all model simulations 3.78%), QALYs - 3.7 to 2.1% (mean - 0.11%), and average cost-utility ratios - 3.0 to 17.9% (mean 4.28%). Applying different published criteria, an overall reproduction success was observed for three of four models. Key replication facilitators were input data tables and model diagrams, while missing standard deviations and missing formulas for equations were considered as key hurdles.

Conclusions: This study confirms the feasibility of rebuilding health economic obesity models, but minor to major assumptions were needed to fill reporting gaps. Model replications can help to assess the quality of health economic model documentation and can be used to validate current model reporting practices. Simple changes to actual CHEERS reporting criteria may solve identified replication hurdles.

Fig. 1

Incremental cost-effectiveness results—original vs. reproduction by case study and comparison. BSC best supportive care/usual care, D&E diet and exercise, QALY quality-adjusted life-year, SBT standard behavioral therapy, SBT+list standard behavioral therapy combined with provision of detailed meal plans and corresponding shopping lists, SIB sibutramine