Pearls and pitfalls of structured staging and reporting of rectal cancer on MRI: an international multireader study

Abstract

Objectives: To investigate uniformity and pitfalls in structured radiological staging of rectal cancer.

Methods: Twenty-one radiologists (12 countries) staged 75 rectal cancers on MRI using a structured reporting template. Interobserver agreement (IOA) was calculated as the percentage agreement between readers (categorical variables) and Krippendorff's α (continuous variables). Agreement with an expert consensus served as a surrogate standard of reference to estimate diagnostic accuracy. Polychoric correlation coefficients were used to assess correlations between diagnostic confidence and accuracy (=agreement with expert consensus).

Results: Uniformity to diagnose high-risk (≥cT3 ab) versus low-risk (≤cT3 cd) cT-stage, cN0 versus cN+, lateral nodes and tumour deposits, MRF and sphincter involvement, and solid versus mucinous tumours was high with IOA > 80% in the majority of cases (and >80% agreement with expert consensus). Results for assessing extramural vascular invasion, cT-stage (cT1-2/cT3/cT4a/cT4b), cN-stage (cN0/N1/N2), relation to the peritoneal reflection, extent of sphincter involvement (internal/intersphincteric/external) and morphology (solid/annular/semi-annular) were considerably poorer. IOA was high (α = 0.72-0.84) for tumour height/length and extramural invasion depth, but low for tumour-MRF distance and number of (suspicious) nodes (α = 0.05-0.55). There was a significant positive correlation between diagnostic confidence and accuracy (=agreement with expert consensus) (p < 0.001-p = 0.003).

Conclusions: - Several staging items lacked sufficient reproducibility.- Results for cT- and N-staging g improved when using a dichotomized stratification.- Considering the significant correlation between diagnostic confidence and accuracy, a confidence level may be incorporated into structured reporting for specific items with low reproducibility.

Advances in knowledge: Although structured reporting aims to achieve uniformity in reporting, several items lack sufficient reproducibility and might benefit from dichotomized assessment and incorporating confidence levels.

Figure 1.

Example of two cases that were scored as semi-annular by the two expert readers. There was large variation among the study readers. Case a was scored by 29% as annular, by 48% as polypoid, and by 24% as semi-annular. Case b was scored by 10% as annular, by 43% as polypoid, and by 48% as semi-annular.

Figure 2.

Example of a case that was staged as cT4a by expert consensus because of involvement of the peritoneum on the left anterior side, above the level of the anterior peritoneal reflection (arrowheads in a). There is simultaneous focal involvement of the MRF below the level of the anterior peritoneal reflection (arrow in b). The study readers unanimously scored this case as MRF+ (100%), but 81% of readers scored it as cT3 and only 19% as T4a.

Figure 3.

Sagittal image (a) showing a stenosing tumour in the mid-rectum with corresponding cross-section (b) at the mid-tumour level. Adjacent to the tumour, there is some desmoplastic stranding as well as several small vessels that radiate outward from the edge of the muscularis propria into the perirectal fat (arrows in B). Because no tumour signal extends into, interrupts or expands these vessels, this case was scored as EMVI-negative by expert consensus. There was however, considerable variation among the study readers; 57% considered this case as EMVI+, 33% as EMVI- and 10% assigned an equivocal score.

Figure 4.

Example of a case with suspect sphincter involvement. The coronal sequence is angled parallel to the distal rectum (rather than to the anal canal), making it more difficult to assess the level of sphincter involvement. The two experts assessed this case as suspicious for intersphincteric involvement, mainly based on the transverse sequence (arrow). Results of the study readers were highly variable; 19% considered it as no sphincter invasion, 19% as internal sphincter invasion only, 19% as extending into the intersphincteric space, and 43% as extending into the external sphincter.

Figure 5.

Examples of two cases where there was substantial variation in tumour-MRF distance among readers. The upper row shows the sagittal (a) and transverse (b) images of a male patient with an upper rectal tumour, situated above the level of the anterior peritoneal reflection. Six out of 21 readers (29%) erroneously interpreted the anterior invasion of the peritoneum as MRF invasion and measured a tumour-MRF distance of 0 mm, while other readers (and the expert reference) measured the distance from the tumour to dorsal MRF as >10 mm. The bottom row (sagittal c, transverse d) shows a male patient with a mid-rectal tumour. Some readers interpreted the anterior rectal wall as involved, while other readers (and the expert reference) believed this tumour was confined to the left dorsolateral wall, resulting in a tumour-MRF distance of >10 mm.