Segmentations of MRI images of the female pelvic floor: a study of inter- and intra-reader reliability

Abstract

Purpose: To describe the inter- and intra-operator reliability of segmentations of female pelvic floor structures.

Materials and methods: Three segmentation specialists were asked to segment out the female pelvic structures in 20 MR datasets on three separate occasions. The STAPLE algorithm was used to compute inter- and intra-segmenter agreement of each organ in each dataset. STAPLE computed the sensitivity, specificity, and positive predictive values (PPV) for inter- and intra-segmenter repeatability. These parameters were analyzed using intra-class correlation analysis. Correlation of organ volume to PPV and sensitivity was also computed.

Results: Mean PPV of the segmented organs ranged from 0.82 to 0.99, and sensitivity ranged from 33 to 96%. Intra-class correlation ranged from 0.07 to 0.98 across segmenters. Pearson correlation of volume to sensitivity were significant across organs, ranging from 0.54 to 0.91. Organs with significant correlation of PPV to volume were bladder (-0.69), levator ani (-0.68), and coccyx (-0.63).

Conclusion: Undirected manual segmentation of the pelvic floor organs are adequate for locating the organs, but poor at defining structural boundaries.

Figure 1

Three cases of high (A,B) and low (C) PPV, with varying sensitivities. The heavy red box represents the consensus segmentation, and A, B, C boxes represent individual segmentations compared against the consensus. The sensitivities vary from low (B) to medium (A) despite high PPV, and a high sensitivity is noted in (C), despite a low relatively low PPV.

Figure 2

a–d: A sample T2-weighted axial MRI slice, taken at the level of the bladder neck is given in Figure 2a. Example individual segmentations of all organs from each of the three readers is given in Figure 2b–d. Legend: red, obturator internus; green, levator ani; blue, vagina; white, rectum; gray, bladder neck; violet, symphysis; pink, pelvic bones.

Figure 3

The T2-weighted axial MRI slice from Figure 1, shaded with the consensus segmentation of each organ as computed by STAPLE. Legend: red, obturator internus; green, levator ani; blue, vagina; white, rectum; gray, bladder neck; violet, symphysis; pink, pelvic bones.

Figure 4

a,b: The distribution of average overlap is given in Figure 3a for a structure with high agreement (the obturator internus), and Figure 3b for a structure with low agreement (levator ani). In these figures, areas in red indicate areas of complete agreement among all nine segmentations for that slice. Orange indicates a region of less agreement, with decreasing agreement in the order yellow, green, and blue, which indicates the least agreement.