Evaluating Artificial Intelligence-Assisted Prostate Biparametric MRI Interpretation: An International Multireader Study

Abstract

BACKGROUND. Variability in prostate biparametric MRI (bpMRI) interpretation limits diagnostic reliability for prostate cancer (PCa). Artificial intelligence (AI) has the potential to reduce this variability and improve diagnostic accuracy. OBJECTIVE. The objective of this study was to evaluate the impact of a deep learning AI model on lesion- and patient-level rates of detection of PCa and clinically significant PCa (csPCa) and interreader agreement for bpMRI interpretations. METHODS. This retrospective, multireader, multicenter study used a balanced incomplete block design for MRI randomization. Six radiologists of varying experience interpreted bpMRI scans with and without AI assistance in alternating sessions. The reference standard for lesion-level detection for cases was whole-mount pathology after radical prostatectomy; for control patients, it was negative 12-core systematic biopsies. In all, 180 patients (120 in the case group and 60 in the control group) who underwent mpMRI and prostate biopsy or radical prostatectomy between January 2013 and December 2022 were included. Lesion-level sensitivity, PPV, and patient-level AUC for csPCa and PCa detection and interreader agreement for lesion-level PI-RADS scores and size measurements were assessed. RESULTS. AI assistance improved lesion-level PPV (PI-RADS ≥ 3: 77.2% [95% CI, 71.0-83.1%] vs 67.2% [95% CI, 61.1-72.2%] for csPCa; 80.9% [75.2-85.7%] vs 69.4% [95% CI, 63.4-74.1%] for PCa; both p < .001), reduced lesion-level sensitivity (PI-RADS ≥ 3: 44.4% [95% CI, 38.6-50.5%] vs 48.0% [95% CI, 42.0-54.2%] for csPCa; p = .01; 41.7% [95% CI, 37.0-47.4%] vs 44.9% [95% CI, 40.5-50.2%] for PCa; p = .01), and no difference in patient-level AUC (0.822 [95% CI, 0.768-0.866] vs 0.832 [95% CI, 0.787-0.868] for csPCa; p = .61; 0.833 [0.782-0.874] vs 0.835 [95% CI, 0.792-0.871] for PCa; p = .91). AI assistance improved interreader agreement for lesion-level PI-RADS scores (κ = 0.748 [95% CI, 0.701-0.796] vs 0.336 [95% CI, 0.288-0.381]; p < .001), lesion size measurements (coverage probability of 0.397 [95% CI, 0.376-0.419] vs 0.367 [95% CI, 0.349-0.383]; p < .001), and patient-level PI-RADS scores (κ = 0.704 [95% CI, 0.627-0.767] vs 0.507 [95% CI, 0.421-0.584]; p < .001). CONCLUSION. AI improved lesion-level PPV and interreader agreement with slightly lower lesion-level sensitivity. CLINICAL IMPACT. AI may enhance consistency and reduce false-positives in bpMRI interpretations. Further optimization is required to improve sensitivity without compromising specificity.

Keywords: MRI; artificial intelligence; computer-aided diagnosis; multireader; prostate cancer.