Machine learning prediction models for stroke-associated pneumonia:Meta-analysis

Abstract

Objective: The heterogeneity of machine learning (ML) models predicting the risk of stroke-associated pneumonia (SAP) is considerable. This study aims to conduct a meta-analysis and comparison of published ML models that predict SAP risk.

Methods: A systematic search was conducted across eight databases, covering the period from their inception to August 16, 2024. Data extraction was performed based on the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) framework. The Assess the Risk of Bias and Applicability of Prediction Model (PROBAST) tool was used to evaluate the risk of bias and applicability of the included models. Descriptive analysis was performed on the included studies, and Meta-Disc 1.4 and Stata 14.0 software were used for sensitivity analysis, subgroup analysis, and meta-regression.

Results: A total of 18 studies comprising 46 SAP risk prediction models were included. The overall Area Under the Curve (AUC) was 0.8623, with a pooled sensitivity of 0.77 (95 % CI: 0.76-0.77, P < 0.001, I² = 94.7 %) and a pooled specificity of 0.75 (95 % CI: 0.74-0.75, P < 0.001, I² = 99.1 %). Logistic regression (LR) was the most commonly used ML method for SAP prediction, with an AUC of 0.8684, sensitivity of 0.77 (95 % CI: 0.75-0.78, P < 0.001, I² = 94.7 %), and specificity of 0.74 (95 % CI: 0.73-0.74, P < 0.001, I² = 98.6 %). In contrast, non-LR models had an AUC of 0.8591, sensitivity of 0.77 (95 % CI: 0.76-0.78, P < 0.001, I² = 94.9 %), and specificity of 0.75 (95 % CI: 0.75-0.75, P < 0.001, I² = 99.3 %). Sensitivity analysis indicated that the random-effects meta-analysis yielded an AUC of 0.8476, sensitivity of 0.77 (95 % CI: 0.76-0.78, P < 0.001, I² = 93.5 %), and specificity of 0.72 (95 % CI: 0.72-0.72, P < 0.001, I² = 98.1 %). Meta-regression analysis revealed that country/region, ML algorithms, participants, year, study source, and study design were not sources of heterogeneity (P = 0.183).

Conclusion: In the existing SAP prediction models, the LR model demonstrates relatively better prediction performance due to its good interpretability and adaptability to smaller sample sizes. However, there are significant limitations in the current research: the overall bias risk of the models is relatively high, the variable handling methods are inconsistent, and there is a scarcity of prediction studies for patients with hemorrhagic stroke. Moreover, the models generally lack external validation, which limits their clinical generalization ability. Future research should conduct prospective, multi-center data studies and carry out internal and external validations to enhance reliability. Strictly following the requirements of CHARMS and PROBAST will effectively reduce the bias risk, enhance the validation efficacy of the models and the clinical translation value.

Keywords: Prediction model; Stroke-associated pneumonia; machine learning.