Interreader Variability of Prostate Imaging Reporting and Data System Version 2 in Detecting and Assessing Prostate Cancer Lesions at Prostate MRI

Abstract

OBJECTIVE. The purpose of this study was to evaluate agreement among radiologists in detecting and assessing prostate cancer at multiparametric MRI using Prostate Imaging Reporting and Data System version 2 (PI-RADSv2). MATERIALS AND METHODS. Treatment-naïve patients underwent 3-T multipara-metric MRI between April 2012 and June 2015. Among the 163 patients evaluated, 110 underwent prostatectomy after MRI and 53 had normal MRI findings and transrectal ultrasound-guided biopsy results. Nine radiologists participated (three each with high, intermediate, and low levels of experience). Readers interpreted images of 58 patients on average (range, 56-60) using PI-RADSv2. Prostatectomy specimens registered to MRI were ground truth. Interob-server agreement was evaluated with the index of specific agreement for lesion detection and kappa and proportion of agreement for PI-RADS category assignment. RESULTS. The radiologists detected 336 lesions. Sensitivity for index lesions was 80.9% (95% CI, 75.1-85.9%), comparable across reader experience (p = 0.392). Patient-level specificity was experience dependent; highly experienced readers had 84.0% specificity versus 55.2% for all others (p < 0.001). Interobserver agreement was excellent for detecting index lesions (index of specific agreement, 0.871; 95% CI, 0.798-0.923). Agreement on PI-RADSv2 category assignment of index lesions was moderate (κ = 0.419; 95% CI, 0.238-0.595). For individual category assignments, proportion of agreement was slight for PI-RADS category 3 (0.208; 95% CI, 0.086-0.284) but substantial for PI-RADS category 4 (0.674; 95% CI, 0.540-0.776). However, proportion of agreement for T2-weighted PI-RADS 4 in the transition zone was 0.250 (95% CI, 0.108-0.372). Proportion of agreement for category assignment of index lesions on dynamic contrast-enhanced MR images was 0.822 (95% CI, 0.728-0.903), on T2-weighted MR images was 0.515 (95% CI, 0.430-0623), and on DW images was 0.586 (95% CI, 0.495-0.682). Proportion of agreement for dominant lesion was excellent (0.828; 95% CI, 0.742-0.913). CONCLUSION. Radiologists across experience levels had excellent agreement for detecting index lesions and moderate agreement for category assignment of lesions using PI-RADS. Future iterations of PI-RADS should clarify PI-RADS 3 and PI-RADS 4 in the transition zone.

Keywords: PI-RADS version 2; agreement; multiparametric MRI; prostate cancer.

Fig. 1—

Histogram shows agreement on lesion detection. Readers had excellent agreement for detecting index lesions using Prostate Imaging Reporting and Data System version 2 criteria. More variability for lesion detection was found in transition zone.

Fig. 2—

Histogram shows agreement on scoring detected lesions by use of Prostate Imaging Reporting and Data System (PI-RADS) version 2. Readers agreed on exact category assignment of 61% of detected lesions, but assignment of PI-RADS category 3 to lesions had poor agreement across reader experience.

Fig. 3—

Histogram shows agreement by pulse sequence and Prostate Imaging Reporting and Data System (PI-RADS) category assignment. T2-weighted sequence, which was dominant in transition zone (TZ) according to PI-RADS, had poor agreement for PI-RADS category 4 lesions. PZ = peripheral zone.