Predicting outcomes after moderate and severe traumatic brain injury using artificial intelligence: a systematic review

Abstract

Methodological standards of existing clinical AI research remain poorly characterized and may partially explain the implementation gap between model development and meaningful clinical translation. This systematic review aims to identify AI-based methods to predict outcomes after moderate to severe traumatic brain injury (TBI), where prognostic uncertainty is highest. The APPRAISE-AI quantitative appraisal tool was used to evaluate methodological quality. We identified 39 studies comprising 592,323 patients with moderate to severe TBI. The weakest domains were methodological conduct (median score 35%), robustness of results (20%), and reproducibility (35%). Higher journal impact factor, larger sample size, more recent publication year and use of data collected in high-income countries were associated with higher APPRAISE-AI scores. Most models were trained or validated using patient populations from high-income countries, underscoring the lack of diverse development datasets and possible generalizability concerns applying models outside these settings. Given its recent development, the APPRAISE-AI tool requires ongoing measurement property assessment.

Fig. 1. Absolute performance difference between AI and non-AI models across c-index (AUC), accuracy, sensitivity and specificity (reported as %).

Positive absolute performance difference values mean AI model performance was higher than non-AI model for the given metric. Stratification corresponds to study-specific APPRAISE-AI score (low, moderate or high). A, B depict results for studies predicting mortality and functional outcome respectively. Note: absolute performance differences reflect comparisons of study-specific performance point estimates, not confidence intervals, which were inconsistently reported in included studies and models. Listed comparisons between AI and non-AI models may therefore overstate performance differences due to unreported confidence intervals quantifying uncertainty. Pease 2022 accuracy, sensitivity and specificity results from AI compared to average of three human experts (neurosurgeons).

Fig. 2. Box plot depicting consensus APPRAISE-AI domain-specific scores, and overall scores determined from review of included studies (n = 39).

Scores were normalized as a proportion of the maximum domain-specific or overall score (percentages). Vertical bars show median values, boxes demonstrate interquartile range (25th to 75th percentile) and whiskers the bounds of 5th and 95th percentiles. Outliers are shown as individual points.

Fig. 3. Box plot depicting individual APPRAISE-AI item-specific scores across study components determined from review of included studies (n = 39).

Scores were normalized as a proportion of the maximum item-specific score (percentages). Vertical bars show median values, boxes demonstrate interquartile range (25th to 75th percentile; no range shown if score distribution for item is narrow) and whiskers the bounds of 5th and 95th percentiles. Outliers are shown as individual points.