The accuracy and quality of image-based artificial intelligence for muscle-invasive bladder cancer prediction

Abstract

Purpose: To evaluate the diagnostic performance of image-based artificial intelligence (AI) studies in predicting muscle-invasive bladder cancer (MIBC). (2) To assess the reporting quality and methodological quality of these studies by Checklist for Artificial Intelligence in Medical Imaging (CLAIM), Radiomics Quality Score (RQS), and Prediction model Risk of Bias Assessment Tool (PROBAST).

Materials and methods: We searched Medline, Embase, Web of Science, and The Cochrane Library databases up to October 30, 2023. The eligible studies were evaluated using CLAIM, RQS, and PROBAST. Pooled sensitivity, specificity, and the diagnostic performances of these models for MIBC were also calculated.

Results: Twenty-one studies containing 4256 patients were included, of which 17 studies were employed for the quantitative statistical analysis. The CLAIM study adherence rate ranged from 52.5% to 75%, with a median of 64.1%. The RQS points of each study ranged from 2.78% to 50% points, with a median of 30.56% points. All models were rated as high overall ROB. The pooled area under the curve was 0.85 (95% confidence interval (CI) 0.81-0.88) for computed tomography, 0.92 (95% CI 0.89-0.94) for MRI, 0.89 (95% CI 0.86-0.92) for radiomics and 0.91 (95% CI 0.88-0.93) for deep learning, respectively.

Conclusion: Although AI-powered muscle-invasive bladder cancer-predictive models showed promising performance in the meta-analysis, the reporting quality and the methodological quality were generally low, with a high risk of bias.

Critical relevance statement: Artificial intelligence might improve the management of patients with bladder cancer. Multiple models for muscle-invasive bladder cancer prediction were developed. Quality assessment is needed to promote clinical application.

Key points: Image-based artificial intelligence models could aid in the identification of muscle-invasive bladder cancer. Current studies had low reporting quality, low methodological quality, and a high risk of bias. Future studies could focus on larger sample sizes and more transparent reporting of pathological evaluation, model explanation, and failure and sensitivity analyses.

Keywords: Artificial intelligence; Magnetic resonance imaging; Muscle-invasive bladder neoplasms; Neoplasm staging; Urinary bladder neoplasms.

Fig. 1

Systematic review flow diagram designed according to PRISMA

Fig. 2

Overview of study characteristics. A Aggregate number of patients included in the study; B Year of publication; C Data collection strategy; D Data source; E Internal validation method; F External validation method

Fig. 3

Diagram showing reporting quality, methodological quality, and risk of bias of each study. The x-axis is the CLAIM adherence rate. The y-axis is the number of yes or probably yes in the PROBAST evaluation. The size of each point is the RQS points

Fig. 4

Diagram showing reporting quality by year and sample size. The x-axis is the year of publication. The y-axis is the CLAIM adherence rate. The size of each point is the sample size

Fig. 5

Diagram showing methodological quality by year and sample size. The x-axis is the year of publication. The y-axis is the RQS points. The size of each point is the sample size

Fig. 6

Diagram showing the risk of bias by year and sample size. The x-axis is the year of publication. The y-axis is the number of yes or probably yes in the PROBAST evaluation. The size of each point is the sample size