How to conduct a systematic review and meta-analysis of prognostic model studies

Abstract

Background: Prognostic models are typically developed to estimate the risk that an individual in a particular health state will develop a particular health outcome, to support (shared) decision making. Systematic reviews of prognostic model studies can help identify prognostic models that need to further be validated or are ready to be implemented in healthcare.

Objectives: To provide a step-by-step guidance on how to conduct and read a systematic review of prognostic model studies and to provide an overview of methodology and guidance available for every step of the review progress.

Sources: Published, peer-reviewed guidance articles.

Content: We describe the following steps for conducting a systematic review of prognosis studies: 1) Developing the review question using the Population, Index model, Comparator model, Outcome(s), Timing, Setting format, 2) Searching and selection of articles, 3) Data extraction using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist, 4) Quality and risk of bias assessment using the Prediction model Risk Of Bias ASsessment (PROBAST) tool, 5) Analysing data and undertaking quantitative meta-analysis, and 6) Presenting summary of findings, interpreting results, and drawing conclusions. Guidance for each step is described and illustrated using a case study on prognostic models for patients with COVID-19.

Implications: Guidance for conducting a systematic review of prognosis studies is available, but the implications of these reviews for clinical practice and further research highly depend on complete reporting of primary studies.

Keywords: Meta-analysis; Prediction model; Prognosis; Prognostic model; Systematic review.

Fig. 1

Review steps. References: the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies [23], Population, Index model, Comparartor model, Outcome(s) model, Timing, Setting [30], search filters [31,32], Prediction model Risk Of Bias ASsessment tool [24,33], meta-analysis [30,34], the Preferred Items for Systematic Reviews and Meta-analyses guidelines [35], Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement [36,37].

Fig. 2

Risk of bias as assessed using the Prediction model Risk Of Bias Assessment tool. The figure represents the percentage of studies scoring a low (green), high (red), or unclear (orange) risk of bias for each of the four Prediction model Risk Of Bias ASsessment domains and the overall risk of bias.

Fig. 3

Forest plots of the Observed Expected ratio and c-statistic of the Pooled Cohort Equations for predicting 10-year risk of cardiovascular disease in women in the general population. ∗Performance of the model in the development study after internal validation. The first row contains the performance of the White model, the second the African American model (not included in the pooled estimate of performance).