Recommendations for prediction models in clinical practice guidelines for cardiovascular diseases are over-optimistic: a global survey utilizing a systematic literature search

Abstract

Background: This study aimed to synthesize the recommendations for prediction models in cardiovascular clinical practice guidelines (CPGs) and assess the methodological quality of the relevant primary modeling studies.

Methods: We performed a systematic literature search of all available cardiovascular CPGs published between 2018 and 2023 that presented specific recommendations (whether in support or non-support) for at least one multivariable clinical prediction model. For the guideline-recommended models, the assessment of the methodological quality of their primary modeling studies was conducted using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

Results: In total, 46 qualified cardiovascular CPGs were included, with 69 prediction models and 80 specific recommendations. Of the 80 specific recommendations, 74 supported 57 models (53 were fully recommended and 4 were conditionally recommended) in cardiovascular practice with moderate to strong strength. Most of the guideline-recommended models were focused on predicting prognosis outcomes (53/57, 93%) in primary and tertiary prevention, focusing primarily on long-term risk stratification and prognosis management. A total of 10 conditions and 7 types of target population were involved in the 57 models, while heart failure (14/57, 25%) and a general population with or without cardiovascular risk factor(s) (12/57, 21%) received the most attention from the guidelines. The assessment of the methodological quality of 57 primary studies on the development of the guideline-recommended models revealed that only 40% of the modeling studies had a low risk of bias (ROB). The causes of high ROB were mainly in the analysis and participant domains.

Conclusions: Global cardiovascular CPGs presented an unduly positive appraisal of the existing prediction models in terms of ROB, leading to stronger recommendations than were warranted. Future cardiovascular practice may benefit from well-established clinical prediction models with better methodological quality and extensive external validation.

Keywords: cardiovascular diseases; clinical practice guideline; methodological quality; prediction model; recommendation; risk of bias.

Figure 1

PRISMA flow diagram of the selection of guidelines.

Figure 2

Classification of recommendations for prediction models. The numbers in parentheses represent the sum total of the guidelines with the same classification of recommendation for a specific model. AHA, aortic dissection risk score; ADHERE, acute decompensated heart failure national registry; ARIC, atherosclerosis risk in communities; ARVC, arrhythmogenic right ventricular cardiomyopathy; CARPREG, cardiac disease in pregnancy study; CART, classification and regression tree; CHADS2, congestive heart failure, hypertension, age >75 years, diabetes, stroke/transient ischemic attack; CHA₂DS₂-VASc, congestive heart failure, hypertension, age ≥75 years (2 points), diabetes mellitus, stroke (2 points)-vascular disease, age 65–74 years, sex category (female); CHARM, candesartan in heart failure-assessment of reduction in mortality and morbidity; COMPASS-CAT, prospective comparison of methods for thromboembolic risk assessment with clinical perceptions and awareness in real-life patients-cancer associated thrombosis; CORONA, controlled rosuvastatin multinational trial in heart failure; EFFECT, enhanced feedback for effective cardiac treatment; ESCAPE, evaluation study of congestive heart failure and pulmonary artery catheterization effectiveness; FRS, Framingham risk score; GO-FAR, good outcome following attempted resuscitation; GRACE, global registry of acute coronary events; GUIDE-IT, guiding evidence-based therapy using biomarker intensified treatment; GWTG-HF, get with the guidelines—heart failure; HAS-BLED, hypertension, abnormal renal/liver function, stroke, bleeding history or predisposition, labile international normalized ratio, elderly (>65 years), drugs/alcohol concomitantly; HCM, hypertrophic cardiomyopathy; HF, heart failure; HF-ACTION, heart failure: a controlled trial investigating outcomes of exercise training; HFSA, Heart Failure Society of America; I-PRESERVE, irbesartan in heart failure with preserved ejection fraction study; J-CTO, multicenter chronic total occlusion registry in Japan; KRS, Khorana risk score; LQTS, long QT syndrome; MAGGIC, meta-analysis global group in chronic heart failure; PARADIGM-HF, prospective comparison of ARNI with ACEI to determine impact on global mortality and morbidity in heart failure trial; PCE, pooled cohort equations; PCP-HF, pooled cohort equations to prevent HF; PERC, pulmonary embolism rule-out criteria; POSSUM, physiological and operative severity score for enumeration of mortality; PROCAM, prospective cardiovascular Münster; PROMISE, prospective multicenter imaging study for evaluation of chest pain; RAM, risk assessment model; RCRI, revised cardiac risk index; SCD, sudden cardiac death; SCORE, systematic coronary risk estimation; SCORE2, systematic coronary risk estimation 2; SCORE2-OP, systematic coronary risk estimation 2-older persons; STS, Society of Thoracic Surgeons; SYNTAX, synergy between percutaneous coronary intervention with TAXUS and cardiac surgery; TAVR, transcatheter aortic valve replacement; TIMI, thrombolysis in myocardial infarction; TOPCAT, treatment of preserved cardiac function heart failure with an aldosterone antagonist trial; TVT, transcatheter valve therapy; TwCCCC, Taiwan Chin-Shan community cardiovascular cohort; VBHOM, vascular biochemical and haematological outcome model; VGNW, vascular governance North West; Vienna-CATS, Vienna cancer and thrombosis study.

Figure 3

The recommended and non-recommended target populations/conditions in clinical practice of the prediction models.

Figure 4

ROB of each primary modeling study.

Figure 5

ROB of the primary studies on guideline-recommended prediction models.