Tufts PACE Clinical Predictive Model Registry: update 1990 through 2015

Abstract

Background: Clinical predictive models (CPMs) estimate the probability of clinical outcomes and hold the potential to improve decision-making and individualize care. The Tufts Predictive Analytics and Comparative Effectiveness (PACE) CPM Registry is a comprehensive database of cardiovascular disease (CVD) CPMs. The Registry was last updated in 2012, and there continues to be substantial growth in the number of available CPMs.

Methods: We updated a systematic review of CPMs for CVD to include articles published from January 1990 to March 2015. CVD includes coronary artery disease (CAD), congestive heart failure (CHF), arrhythmias, stroke, venous thromboembolism (VTE), and peripheral vascular disease (PVD). The updated Registry characterizes CPMs based on population under study, model performance, covariates, and predicted outcomes.

Results: The Registry includes 747 articles presenting 1083 models, including both prognostic (n = 1060) and diagnostic (n = 23) CPMs representing 183 distinct index condition/outcome pairs. There was a threefold increase in the number of CPMs published between 2005 and 2014, compared to the prior 10-year interval from 1995 to 2004. The majority of CPMs were derived from either North American (n = 455, 42%) or European (n = 344, 32%) populations. The database contains 265 CPMs predicting outcomes for patients with coronary artery disease, 196 CPMs for population samples at risk for incident CVD, and 158 models for patients with stroke. Approximately two thirds (n = 701, 65%) of CPMs report a c-statistic, with a median reported c-statistic of 0.77 (IQR, 0.05). Of the CPMs reporting validations, only 333 (57%) report some measure of model calibration. Reporting of discrimination but not calibration is improving over time (p for trend < 0.0001 and 0.39 respectively).

Conclusions: There is substantial redundancy of CPMs for a wide spectrum of CVD conditions. While the number of CPMs continues to increase, model performance is often inadequately reported and calibration is infrequently assessed. More work is needed to understand the potential impact of this literature.

Keywords: Cardiovascular disease risk factors; Cerebrovascular disease/stroke; Clinical predictive model; Coronary artery disease; Methods; Modeling; Prediction; Prognostic factor; Risk stratification.

Fig. 1

PubMed was searched for relevant articles from 1990 to March 2015

Fig. 2

Cumulative growth in published CPM articles included in the Tufts CPM database over time (January 1990–March 2015). Dark blue represents models derived on CVD-free population samples. Light blue represents models derived on patients with specific cardiovascular conditions at baseline

Fig. 3

CPMs by derivation cohort geographic region

Fig. 4

Discrimination of CPMs by index condition. Discrimination is reported as c-statistic (median, IQR). CPM, clinical prediction model; CAD, coronary artery disease; CHF, congestive heart failure; VTE, venous thromboembolism

Fig. 5

Frequency of covariate categories among all covariates (n = 9641) in the Tufts PACE CPM Registry. Top covariates across the top 5 index conditions are also presented