Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Sep;47(9):3318-3326.
doi: 10.1007/s00261-022-03580-8. Epub 2022 Jun 28.

Interobserver agreement between eight observers using IOTA simple rules and O-RADS lexicon descriptors for adnexal masses

Neha Antil  1 , Preethi R Raghu  2 , Luyao Shen  1 , Thodsawit Tiyarattanachai  1 , Edwina M Chang  3 , Craig W K Ferguson  4 , Amanzo A Ho  1 , Amelie M Lutz  1 , Aladin J Mariano  1 , L Nayeli Morimoto  1 , Aya Kamaya  1
Affiliations

Interobserver agreement between eight observers using IOTA simple rules and O-RADS lexicon descriptors for adnexal masses

Neha Antil et al. Abdom Radiol (NY). 2022 Sep.

Abstract

Purpose: To evaluate interobserver agreement in assigning imaging features and classifying adnexal masses using the IOTA simple rules versus O-RADS lexicon and identify causes of discrepancy.

Methods: Pelvic ultrasound (US) examinations in 114 women with 118 adnexal masses were evaluated by eight radiologists blinded to the final diagnosis (4 attendings and 4 fellows) using IOTA simple rules and O-RADS lexicon. Each feature category was analyzed for interobserver agreement using intraclass correlation coefficient (ICC) for ordinal variables and free marginal kappa for nominal variables. The two-tailed significance level (a) was set at 0.05.

Results: For IOTA simple rules, interobserver agreement was almost perfect for three malignant lesion categories (M2-4) and substantial for the remaining two (M1, M5) with k-values of 0.80-0.82 and 0.68-0.69, respectively. Interobserver agreement was almost perfect for two benign feature categories (B2, B3), substantial for two (B4, B5) and moderate for one (B1) with k-values of 0.81-0.90, 0.69-0.70 and 0.60, respectively. For O-RADS, interobserver agreement was almost perfect for two out of ten feature categories (ascites and peritoneal nodules) with k-values of 0.89 and 0.97. Interobserver agreement ranged from fair to substantial for the remaining eight feature categories with k-values of 0.39-0.61. Fellows and attendings had ICC values of 0.725 and 0.517, respectively.

Conclusion: O-RADS had variable interobserver agreement with overall good agreement. IOTA simple rules had more uniform interobserver agreement with overall excellent agreement. Greater reader experience did not improve interobserver agreement with O-RADS.

Keywords: Adnexal; IOTA; O-RADS; Ovarian; Pelvic; Ultrasound.

PubMed Disclaimer

Conflict of interest statement

All authors declare that they have no conflict of interest.

Figures

Fig. 1
Fig. 1
Flowchart on deriving the final cohort
Fig. 2
Fig. 2
Ultrasound Examples of IOTA Features. A Grayscale image of an ovarian multilocular cystic lesion with multiple (≥ 4) papillary projections (white arrows). This is an example of M3: ≥ 4 papillary projections, which had almost perfect interobserver agreement. B Grayscale image of an ovarian echogenic lesion with acoustic shadows (example of B3), which had almost perfect interobserver agreement. C Color Doppler image of an ovarian solid lesion with very strong blood flow (example of M5), which had substantial interobserver agreement. D Grayscale image of an ovarian unilocular cyst (example of B1), which had moderate interobserver agreement
Fig. 3
Fig. 3
Ultrasound Examples of O-RADS Lexicon. A Grayscale image of an ovarian cystic lesion with solid component. This is an example of lesion type, which had substantial interobserver agreement. B Color Doppler image of an ovarian cystic lesion with solid component. This is an example of color score 2 (mild), which had good interobserver agreement. C Grayscale image of ovarian cyst with a smooth septation. This is an example of septation type, which had moderate interobserver agreement. D Grayscale image of a complex ovarian cystic lesion with white arrow denoting the irregular inner wall. This is an example of inner wall, which had fair interobserver agreement
See this image and copyright information in PMC

Comment in

  • Letter to the Editor.
    Gunes YC. Gunes YC. Abdom Radiol (NY). 2024 Oct;49(10):3758. doi: 10.1007/s00261-024-04220-z. Epub 2024 Jun 24. Abdom Radiol (NY). 2024. PMID: 38913138 No abstract available.

References

    1. Timmerman D, Testa AC, Bourne T, Ameye L, Jurkovic D, Van Holsbeke C, et al. Simple ultrasound-based rules for the diagnosis of ovarian cancer. Ultrasound Obstet Gynecol. 2008;31(6):681–690. doi: 10.1002/uog.5365. - DOI - PubMed
    1. Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, Van Holsbeke C, Savelli L, Fruscio R, Lissoni AA, Testa AC, Veldman J, Vergote I, Van Huffel S, Bourne T, Valentin L. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ. 2010;14(341):c6839. doi: 10.1136/bmj.c6839. - DOI - PMC - PubMed
    1. Timmerman D, Van Calster B, Testa A, Savelli L, Fischerova D, Froyman W, et al. Predicting the risk of malignancy in adnexal masses based on the simple rules from the international ovarian tumor analysis group. Am J Obstet Gynecol. 2016;214(4):424–437. doi: 10.1016/j.ajog.2016.01.007. - DOI - PubMed
    1. DePriest PD, Shenson D, Fried A, Hunter JE, Andrews SJ, Gallion HH, Pavlik EJ, Kryscio RJ, van Nagell JR. A morphology index based on sonographic findings in ovarian cancer. Gynecol Oncol. 1993;51(1):7–11. doi: 10.1006/gyno.1993.1238. - DOI - PubMed
    1. Craig JM, Martinez J. Gynecologic imaging reporting and data system: a new proposal for classifying adnexal masses on the basis of sonographic findings. J Ultrasound Med. 2009;28(3):285–291. doi: 10.7863/jum.2009.28.3.285. - DOI - PubMed