Endometriosis diagnosis and staging by operating surgeon and expert review using multiple diagnostic tools: an inter-rater agreement study

Abstract

Objective: To determine agreement on endometriosis diagnosis between real-time laparoscopy and subsequent expert review of digital images, operative reports, magnetic resonance imaging (MRI), and histopathology, viewed sequentially.

Design: Inter-rater agreement study.

Setting: Five urban surgical centres.

Population: Women, aged 18-44 years, who underwent a laparoscopy regardless of clinical indication. A random sample of 105 women with and 43 women without a postoperative endometriosis diagnosis was obtained from the ENDO study.

Methods: Laparoscopies were diagnosed, digitally recorded, and reassessed.

Main outcome measures: Inter-observer agreement of endometriosis diagnosis and staging according to the revised American Society for Reproductive Medicine criteria. Prevalence and bias-adjusted kappa values (κ) were calculated for diagnosis, and weighted κ values were calculated for staging.

Results: Surgeons and expert reviewers had substantial agreement on diagnosis and staging after viewing digital images (n = 148; mean κ = 0.67, range 0.61-0.69; mean κ = 0.64, range 0.53-0.78, respectively) and after additionally viewing operative reports (n = 148; mean κ = 0.88, range 0.85-0.89; mean κ = 0.85, range 0.84-0.86, respectively). Although additionally viewing MRI findings (n = 36) did not greatly impact agreement, agreement substantially decreased after viewing histological findings (n = 67), with expert reviewers changing their assessment from a positive to a negative diagnosis in up to 20% of cases.

Conclusion: Although these findings suggest that misclassification bias in the diagnosis or staging of endometriosis via visualised disease is minimal, they should alert gynaecologists who review operative images in order to make decisions on endometriosis treatment that operative reports/drawings and histopathology, but not necessarily MRI, will improve their ability to make sound judgments.

Tweetable abstract: Endometriosis diagnosis and staging agreement between expert reviewers and operating surgeons was substantial.

Keywords: Endometriosis; epidemiology; histology; laparoscopy; magnetic resonance imaging; reliability.

Figure 1. Endometriosis diagnosis

Unadjusted pair-wise Cohen's kappa (κ) and 95% CI of endometriosis diagnosis by operating surgeon and expert reviewer during real-time laparoscopy in the Endometriosis, Natural History, Diagnosis, and Outcomes Study, Salt Lake City, UT, USA, 2007–09. Prevalence and bias adjusted κ for Reviewers 1–4 after viewing images were 0.69, 0.60, 0.69, and 0.69 respectively; after images + operative report were 0.89, 0.89, 0.86, and 0.85 respectively; after images + operative report + MRI were 0.78, 0.72, 0.81, and 0.88; after images + operative report + histologic findings were 0.69, 0.75, 0.71, and 0.62; and after images + operative report + MRI histologic findings were 0.91, 0.82, 1.00, and 1.00. Note: No confidence intervals when perfect agreement (κ=1.0).

Figure 2. Endometriosis Staging, I-IV (Empiric Assessment)

Cohen's weighted kappa (κ) and 95% CI of endometriosis staging based on rASRM empiric assessment between operating surgeon and expert reviewer during real-time laparoscopy in the Endometriosis, Natural History, Diagnosis, and Outcomes Study, Salt Lake City, UT, USA, 2007–09.

Figure 3. Endometriosis Staging, I-IV (Algorithm Assessment)

Cohen's weighted kappa (κ) and 95% CI of endometriosis staging based on rASRM algorithm assessment between operating surgeon and expert reviewer during real-time laparoscopy in the Endometriosis, Natural History, Diagnosis, and Outcomes Study, Salt Lake City, UT, USA, 2007–09.