The learning curve in bladder MRI using VI-RADS assessment score during an interactive dedicated training program

Abstract

Objective: The purpose of the study was to evaluate the effect of an interactive training program on the learning curve of radiology residents for bladder MRI interpretation using the VI-RADS score.

Methods: Three radiology residents with minimal experience in bladder MRI served as readers. They blindly evaluated 200 studies divided into 4 subsets of 50 cases over a 3-month period. After 2 months, the first subset was reassessed, resulting in a total of 250 evaluations. An interactive training program was provided and included educational lessons and case-based practice. The learning curve was constructed by plotting mean agreement as the ratio of correct evaluations per batch. Inter-reader agreement and diagnostic performance analysis were performed with kappa statistics and ROC analysis.

Results: As for the VI-RADS scoring agreement, the kappa differences between pre-training and post-training evaluation of the same group of cases were 0.555 to 0.852 for reader 1, 0.522 to 0.695 for reader 2, and 0.481 to 0.794 for reader 3. Using VI-RADS ≥ 3 as cut-off for muscle invasion, sensitivity ranged from 84 to 89% and specificity from 91 to 94%, while the AUCs from 0.89 (95% CI:0.84, 0.94) to 0.90 (95% CI:0.86, 0.95). Mean evaluation time decreased from 5.21 ± 1.12 to 3.52 ± 0.69 min in subsets 1 and 5. Mean grade of confidence improved from 3.31 ± 0.93 to 4.21 ± 0.69, in subsets 1 and 5.

Conclusion: An interactive dedicated education program on bladder MRI and the VI-RADS score led to a significant increase in readers' diagnostic performance over time, with a general improvement observed after 100-150 cases.

Key points: • After the first educational lesson and 100 cases were interpreted, the concordance on VI-RADS scoring between the residents and the experienced radiologist was significantly higher. • An increase in the grade of confidence was experienced after 100 cases. • We found a decrease in the evaluation time after 150 cases.

Keywords: Bladder cancer; Internship and residency; Learning curve; Magnetic resonance imaging; Training program.

Fig. 1

Study flow diagram. MIBC, muscle-invasive bladder cancer

Fig. 2

Reader’s learning curve for VI-RADS scoring over time, based on the ratio of concordant evaluations between readers and reference standard, per subset of images. R1, reader 1; R2, reader 2; R3, reader 3; RS, reference standard; RMean, reader’s mean

Fig. 3

ROC curve demonstrating the AUCs for the three readers in detecting bladder cancer muscle invasiveness. ROC, reader characteristics curve; AUC, area under the curve

Fig. 4

Reader’s mean evaluation time for each subset (in minutes). R1, reader 1; R2, reader 2; R3, reader 3; RS, reference standard; RMean, reader’s mean

Fig. 5

Reader’s mean grade of confidence score for each subset (5-point assessment scale). R1, reader 1; R2, reader 2; R3, reader 3; RS, reference standard; RMean, reader’s mean.

Fig. 6

Case example of a 67-year-old male. a Coronal T2WI showing a pedunculated bladder tumor at the bladder dome extending to the right lateral wall with a clearly uninterrupted muscularis propria layer (arrow); b axial T2WI showing an apparently interrupted muscularis propria (arrow); c sagittal T2WI showing an equivocal alteration of the muscularis propria (arrow); d, e DWI and ADC map showing no interruption of the muscularis propria layer (arrows), and the “inchworm sign”, which is usually indicative, as in this case, of NMIBC; f DCE MRI showing the integrity of the muscularis propria layer and the inner layer sign (arrow), both indicative of NMIBC. The images were incorrectly scored as a VI-RADS 3 and 4 by the inexperienced readers during the first interpretation batch, probably due to the non-optimal quality acquisition; however, MRI was correctly scored with an overall VI-RADS 2 in batch 5, given the higher reader experience. T stage after TURBT identified HG-T1 urothelial carcinoma. T2WI, T2-weighted imaging; VI-RADS, Vesical Imaging-Reporting and Data System; DWI, diffusion-weighted imaging; ADC, apparent diffusion coefficient; DCE, dynamic contrast-enhanced; NMIBC, non-muscle-invasive bladder cancer; TURBT, trans-urethral resection of bladder tumor; HG, high-grade