Interobserver Variability of Sonographic Features Used in the American College of Radiology Thyroid Imaging Reporting and Data System

Abstract

Objective: The purpose of this study was to assess interobserver variability in assigning features in the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) lexicon and in making recommendations for thyroid nodule biopsy.

Materials and methods: The study cohort comprised 100 nodules in 92 patients who underwent fine-needle aspiration with definitive cytologic results (Bethesda category II or VI) or diagnostic lobectomy between April 2009 and May 2010. Eight board-certified radiologists evaluated the nodules according to the five feature categories that constitute ACR TI-RADS and gave a biopsy recommendation based on their own practice. Variability in feature assignment and biopsy recommendation was assessed with the Fleiss kappa statistic.

Results: Agreement in interpretation was fair to moderate for all features except shape (κ = 0.61) and macrocalcifications (κ = 0.73), which had substantial agreement. The features with the poorest agreement were margin and other types of echogenic foci, which had kappa values ranging from 0.25 to 0.39, indicating fair agreement. Interobserver agreement regarding biopsy recommendation was fair (κ = 0.22) based on radiologists' current practice. Applying ACR TI-RADS resulted in moderate agreement (κ = 0.51).

Conclusion: Variability in interpreting thyroid nodule sonographic features was highest for margin and all types of echogenic foci, except for macrocalcifications. Because radiologists' interpretations of these features change the level of suspicion of thyroid malignancy, the results of this study suggest a need for further education. Despite the variability in assigning features, adoption of ACR TI-RADS improves agreement for recommending biopsy.

Keywords: TI-RADS; interobserver agreement; interobserver variability; thyroid malignancy; thyroid ultrasound.