Prognostic risk models for incident hypertension: A PRISMA systematic review and meta-analysis

Abstract

Objective: Our goal was to review the available literature on prognostic risk prediction for incident hypertension, synthesize performance, and provide suggestions for future work on the topic.

Methods: A systematic search on PUBMED and Web of Science databases was conducted for studies on prognostic risk prediction models for incident hypertension in generally healthy individuals. Study-quality was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST) checklist. Three-level meta-analyses were used to obtain pooled AUC/C-statistic estimates. Heterogeneity was explored using study and cohort characteristics in meta-regressions.

Results: From 5090 hits, we found 53 eligible studies, and included 47 in meta-analyses. Only four studies were assessed to have results with low risk of bias. Few models had been externally validated, with only the Framingham risk model validated more than thrice. The pooled AUC/C-statistics were 0.82 (0.77-0.86) for machine learning models and 0.78 (0.76-0.80) for traditional models, with high heterogeneity in both groups (I2 > 99%). Intra-class correlations within studies were 60% and 90%, respectively. Follow-up time (P = 0.0405) was significant for ML models and age (P = 0.0271) for traditional models in explaining heterogeneity. Validations of the Framingham risk model had high heterogeneity (I2 > 99%).

Conclusion: Overall, the quality of included studies was assessed as poor. AUC/C-statistic were mostly acceptable or good, and higher for ML models than traditional models. High heterogeneity implies large variability in the performance of new risk models. Further, large heterogeneity in validations of the Framingham risk model indicate variability in model performance on new populations. To enable researchers to assess hypertension risk models, we encourage adherence to existing guidelines for reporting and developing risk models, specifically reporting appropriate performance measures. Further, we recommend a stronger focus on validation of models by considering reasonable baseline models and performing external validations of existing models. Hence, developed risk models must be made available for external researchers.

Fig 1. PRISMA diagram of search process and selection of eligible records.

Fig 2. Variables used in studies.

Variables were counted as those used in any final developed model in a study. We summarize by studies due to variation in the number of developed models per study. Variables used by only one single study were either merged with similar ones, or grouped as”Other” within its category. Note: Variable information from five studies were excluded as they did not report complete information, meaning variable information from 45 studies developing new models are included here. ‘BMI’: Body Mass Index, ‘BP’: Blood Pressure,’ Chol.’: Cholesterol, ‘HDL’: High-density lipoprotein, ‘LDL’: Low-density lipoprotein, ‘Misc’: Miscellaneous, ‘SNPs’: Single nucleotide polymorphisms.

Fig 3. ROB assessment using short form PROBAST.

PROBAST assessment summarized per study. In short, domains were: 1) Outcome assessment, 2) EPV, 3) Continuous predictors handling, 4) Missing data management, 5) Univariable selection of predictors, and 6) Correction for overfitting/optimism. See Venema et al. [12] for more details. Domains 3, 5 and 6 were not applicable for external validations. Six studies had remarks that were only valid for some of the reported results, e.g., due to the Events Per Variable (EPV) criteria being less strict for external validations or different methods used on some of the developed models. These were marked with a mixed “High/Low” symbol on relevant domain or overall assessment. ‘Dev.’: Developed models, ‘Ext. val.’: External validations, ‘NA’: Not applicable.

Fig 4. ROB assessment using long form PROBAST.

PROBAST assessment for studies marked potentially low risk of bias using short form PROBAST. ‘An.’: Analysis, ‘App.’: Applicability, ‘DEV’: Developed models, ‘NI’: No information, ‘Out.’: Outcome, ‘Part.’: Participants, ‘Pred.’: Predictors, ‘ROB’: Risk of bias, ‘VAL’: External validations.

Fig 5. Forest plot of traditional models.

The 95% prediction interval for new models extending from the summary diamond on the bottom line was calculated as (0.660–0.865). ‘*’: The result was obtained from a risk score, or nomogram, developed using that method. ‘CI’: Confidence interval, ‘Reg.’: Regression.

Fig 6. Forest plot of ML models.

The 95% prediction interval for new models extending from the summary diamond on the bottom line was calculated as (0.547–0.943). ‘Method 1 + Method 2’: Ensemble of Method 1 and 2. ‘Method 1 into Method 2’: Outputs from Method 1 were used as inputs to Method 2. ‘BN’: Bayes Network, ‘CI’: Confidence interval, ‘DT’: Decision tree, ‘GBM’: Gradient Boosting Machines, ‘KNN’: K-Nearest Neighbor, ‘LSTM NN’: Long Short-Term Memory Neural Net, ‘LWNB’: Locally Weighted Naïve Bayes’, ‘MLP NN’: Multi-Layer Perceptron Neural Net, ‘NB’: Naïve Bayes, ‘Reg.’: Regression, ‘RF’: Random Forest, ‘SVM’: Support Vector Machines, ‘XGBoost’: eXtreme Gradient Boosting.

Fig 7. Forest plot for external validations of the Framingham risk model.

The 95% prediction interval extending from the summary diamond was estimated as (0.571–0.883). ‘CI’: Confidence interval.