Interrater Reliability of NI-RADS on Posttreatment PET/Contrast-enhanced CT Scans in Head and Neck Squamous Cell Carcinoma

Abstract

Purpose To evaluate the interrater reliability among radiologists examining posttreatment head and neck squamous cell carcinoma (HNSCC) fluorodeoxyglucose PET/contrast-enhanced CT (CECT) scans using Neck Imaging Reporting and Data System (NI-RADS). Materials and Methods In this retrospective study, images in 80 patients with HNSCC who underwent posttreatment surveillance PET/CECT and immediate prior comparison CECT or PET/CECT (from June 2014 to July 2016) were uploaded to the American College of Radiology's cloud-based website, Cortex. Eight radiologists from seven institutions with variable NI-RADS experience independently evaluated each case and assigned an appropriate prose description and NI-RADS category for the primary site and the neck site. Five of these individuals were experienced readers (> 5 years of experience), and three were novices (< 5 years of experience). In total, 640 lexicon-based and NI-RADS categories were assigned to lesions among the 80 included patients by the eight radiologists. Light generalization of Cohen κ for interrater reliability was performed. Results Of the 80 included patients (mean age, 63 years ± 10 [standard deviation]), there were 58 men (73%); 60 patients had stage IV HNSCC (75%), and the most common tumor location was oropharynx (n = 32; 40%). Light κ for lexicon was 0.30 (95% CI: 0.23, 0.36) at the primary site and 0.31 (95% CI: 0.24, 0.37) at the neck site. Light κ for NI-RADS category was 0.55 (95% CI: 0.46, 0.63) at the primary site and 0.60 (95% CI: 0.48, 0.69) at the neck site. Percent agreement between lexicon and correlative NI-RADS category was 84.4% (540 of 640) at the primary site and 92.6% (593 of 640) at the neck site. There was no significant difference in interobserver agreement among the experienced versus novice raters. Conclusion Moderate agreement was achieved among eight radiologists using NI-RADS at posttreatment HNSCC surveillance imaging. Keywords: CT, PET/CT, Head/Neck, Neck, Neoplasms-Primary, Observer Performance Supplemental material is available for this article. © RSNA, 2021.

Keywords: CT; Head/Neck; Neck; Neoplasms-Primary; Observer Performance; PET/CT.

Figure 1:

Prose description versus Neck Imaging Reporting and Data System (NI-RADS) category, A, at the primary site and, B, at the neck site.

Figure 2:

T2N0M0 squamous cell carcinoma of the right buccal mucosa treated with wide local excision, selective neck dissection, and right forearm free flap. A, At 3-month posttreatment contrast material–enhanced CT, there was a linear area of enhancement along the lateral aspect of the right first mandibular molar (yellow arrow). At PET, this area was associated with intense fluorodeoxyglucose uptake (yellow arrow). Of note, the curvilinear enhancing lesion represents the pedicle of the forearm free flap (red arrow). Clinical examination findings were not suggestive of tumor recurrence but felt to represent tumefactive radiation injury. B, A contrast-enhanced neck CT scan with bone window obtained 9 months after treatment demonstrates destructive osseous changes centered around the right maxillary tuberosity and right mandible (yellow arrow). CT-guided biopsy revealed fibrotic tissue with inflammation, consistent with osteoradionecrosis. C, At imaging 12 months after treatment, there is interval improvement in fluorodeoxyglucose uptake (yellow arrow). Half of the raters assigned the first posttreatment study a NI-RADS category 3 at the primary site. However, given its mucosal location for direct clinical examination, NI-RADS 2a is more appropriate. Red arrow indicates curvilinear enhancing lesion representing pedicle of forearm free flap.

Figure 3:

A, T4aN1M0 squamous cell carcinoma of the right palatine tonsil staging PET/contrast-enhanced CT (CECT). Patient was treated with chemoradiation therapy alone. B, At 3 months after treatment, PET/CECT scan demonstrated an enhancing soft-tissue mass along the superior margin of ulceration in the region of palatine tonsil with corresponding intense fluorodeoxyglucose uptake, concerning for residual tumor. However, upon direct visualization, the otolaryngologists suspected the imaging findings were radiation injury to soft tissues. A CT-guided biopsy revealed inflammatory cells and necrotic debris. C, On 6-month follow-up images, there is marked improvement in fluorodeoxyglucose uptake. This case was particularly difficult, with half of our raters assigning a NI-RADS category 2a and the others a 3, and became an index case to help define the 2a category. Immediate posttreatment PET/CECT scans with mucosal-based abnormalities are now typically assigned a NI-RADS category 2a, as many of these are tumefactive radiation injury.