Multicenter Multireader Evaluation of an Artificial Intelligence-Based Attention Mapping System for the Detection of Prostate Cancer With Multiparametric MRI

Abstract

OBJECTIVE. The purpose of this study was to evaluate in a multicenter dataset the performance of an artificial intelligence (AI) detection system with attention mapping compared with multiparametric MRI (mpMRI) interpretation in the detection of prostate cancer. MATERIALS AND METHODS. MRI examinations from five institutions were included in this study and were evaluated by nine readers. In the first round, readers evaluated mpMRI studies using the Prostate Imaging Reporting and Data System version 2. After 4 weeks, images were again presented to readers along with the AI-based detection system output. Readers accepted or rejected lesions within four AI-generated attention map boxes. Additional lesions outside of boxes were excluded from detection and categorization. The performances of readers using the mpMRI-only and AI-assisted approaches were compared. RESULTS. The study population included 152 case patients and 84 control patients with 274 pathologically proven cancer lesions. The lesion-based AUC was 74.9% for MRI and 77.5% for AI with no significant difference (p = 0.095). The sensitivity for overall detection of cancer lesions was higher for AI than for mpMRI but did not reach statistical significance (57.4% vs 53.6%, p = 0.073). However, for transition zone lesions, sensitivity was higher for AI than for MRI (61.8% vs 50.8%, p = 0.001). Reading time was longer for AI than for MRI (4.66 vs 4.03 minutes, p < 0.001). There was moderate interreader agreement for AI and MRI with no significant difference (58.7% vs 58.5%, p = 0.966). CONCLUSION. Overall sensitivity was only minimally improved by use of the AI system. Significant improvement was achieved, however, in the detection of transition zone lesions with use of the AI system at the cost of a mean of 40 seconds of additional reading time.

Keywords: MRI; artificial intelligence; laparoscopic; multiparametric; prostate cancer; radical prostatectomy; robot-assisted.

Fig. 1—

55-year-old man with prostate-specific antigen level of 4.68 ng/mL and Prostate Imaging Reporting and Data System category 5 lesion in left anterior transition zone correctly detected by artificial intelligence system. Final histopathologic result was Gleason 3 + 4 prostate cancer. A, T2-weighted MR image. B, Apparent diffusion coefficient map. C, DW image (b = 2000 mm/s²). D, Dynamic contrast-enhanced MR image. E, T2-weighted MR image with attention box produced by means of artificial intelligence. F, Photomicrograph of radical prostatectomy specimen. I = index lesion.

Fig. 2—

Graphs show sensitivity and specificity of artificial intelligence (AI) and MRI for different Prostate Imaging Reporting and Data System (PI-RADS) category thresholds at patient level. Asterisk denotes p < 0.05; double asterisk, p < 0.01. A, Sensitivity for all readers. B, Sensitivity for readers with low level of experience. C, Sensitivity for readers with moderate level of experience. D, Sensitivity for readers with high level of experience. E, Specificity for all readers. F, Specificity for readers with low level of experience. G, Specificity for readers with moderate level of experience. H, Specificity for readers with high level of experience.

Fig. 2—

Graphs show sensitivity and specificity of artificial intelligence (AI) and MRI for different Prostate Imaging Reporting and Data System (PI-RADS) category thresholds at patient level. Asterisk denotes p < 0.05; double asterisk, p < 0.01. A, Sensitivity for all readers. B, Sensitivity for readers with low level of experience. C, Sensitivity for readers with moderate level of experience. D, Sensitivity for readers with high level of experience. E, Specificity for all readers. F, Specificity for readers with low level of experience. G, Specificity for readers with moderate level of experience. H, Specificity for readers with high level of experience.

Fig. 3—

Graphs show sensitivity of artificial intelligence (AI) and MRI for different Prostate Imaging Reporting and Data System (PI-RADS) category thresholds at lesion level for whole prostate, peripheral zone, and transition zone. Asterisk denotes p < 0.05; double asterisk, p < 0.01. A, Whole prostate, all readers. B, Whole prostate, readers with low level of experience. C, Whole prostate, readers with moderate level of experience. D, Whole prostate, readers with high level of experience. E, Peripheral zone, all readers. F, Peripheral zone, readers with low level of experience. G, Peripheral zone, readers with moderate level of experience. H, Peripheral zone, readers with high level of experience. I, Transition zone, all readers. J, Transition zone, readers with low level of experience. K, Transition zone, readers with moderate level of experience. L, Transition zone, readers with high level of experience.

Fig. 3—

Graphs show sensitivity of artificial intelligence (AI) and MRI for different Prostate Imaging Reporting and Data System (PI-RADS) category thresholds at lesion level for whole prostate, peripheral zone, and transition zone. Asterisk denotes p < 0.05; double asterisk, p < 0.01. A, Whole prostate, all readers. B, Whole prostate, readers with low level of experience. C, Whole prostate, readers with moderate level of experience. D, Whole prostate, readers with high level of experience. E, Peripheral zone, all readers. F, Peripheral zone, readers with low level of experience. G, Peripheral zone, readers with moderate level of experience. H, Peripheral zone, readers with high level of experience. I, Transition zone, all readers. J, Transition zone, readers with low level of experience. K, Transition zone, readers with moderate level of experience. L, Transition zone, readers with high level of experience.