RSNA and BSTI grading systems of COVID-19 pneumonia: comparison of the diagnostic performance and interobserver agreement

Abstract

Background: This study aimed to compare the performance and interobservers agreement of cases with findings on chest CT based on the British Society of Thoracic Imaging (BSTI) guideline statement of COVID-19 and the Radiological Society of North America (RSNA) expert consensus statement.

Methods: In this study, 903 patients who had admitted to the emergency department with a pre-diagnosis of COVID-19 between 1 and 18 July 2020 and had chest CT. Two radiologists classified the chest CT findings according to the RSNA and BSTI consensus statements. The performance, sensitivity and specificity values of the two classification systems were calculated and the agreement between the observers was compared by using kappa analysis.

Results: Considering RT-PCR test result as a gold standard, the sensitivity, specificity and positive predictive values were significantly higher for the two observers according to the BSTI guidance statement and the RSNA expert consensus statement (83.3%, 89.7%, 89.0%; % 81.2,% 89.7,% 88.7, respectively). There was a good agreement in the PCR positive group (κ: 0.707; p < 0.001 for BSTI and κ: 0.716; p < 0.001 for RSNA), a good agreement in the PCR negative group (κ: 0.645; p < 0.001 for BSTI and κ: 0.743; p < 0.001 for RSNA) according to the BSTI and RSNA classification between the two radiologists.

Conclusion: As a result, RSNA and BSTI statement provided reasonable performance and interobservers agreement in reporting CT findings of COVID-19. However, the number of patients defined as false negative and indeterminate in both classification systems is at a level that cannot be neglected.

Keywords: Computed tomography; Coronavirus; Guidelines of radiology; RT-PCR.

Fig. 1

Flow chart illustrating the patient selection

Fig. 2

a, b Case examples defined in the classic COVID-19 for the BSTI guidance statement, and in the typical category for the RSNA expert consensus statement. a An axial chest CT image of a 68-year-old male with COVID-19 shows peripheral, bilateral, multifocal GGOs. b An axial chest CT image of a 64-year-old male with COVID-19 shows peripheral, bilateral, multifocal GGOs

Fig. 3

a, b Case examples defined in the probable COVID-19 for the BSTI guidance statement, and in the typical category for the RSNA expert consensus statement. a An axial chest CT image of a 46-year-old female with COVID-19 shows peripheral, bilateral, multifocal, rounded GGOs (arrows). b An axial chest CT image of a 19-year-old male with COVID-19 shows peripheral, bilateral, multifocal, rounded GGOs (arrows)

Fig. 4

a, b Case examples defined in the classic COVID-19 for the BSTI guidance statement, and in the indeterminate category for the RSNA expert consensus statement. a An axial chest CT image of a 65-year-old female with COVID-19 shows segmental (the left upper lobe) GGOs and bronchial wall thickening. b An axial chest CT image of a 41-year-old female with COVID-19 shows segmental GGOs and vascular enlargement with bronchial wall thickening (arrows)

Fig. 5

a, b Case examples defined in the non-COVID for the BSTI guidance statement, and in the atypical category for the RSNA expert consensus statement. a An axial chest CT image of a 36-year-old male without COVID-19 shows centrilobular nodules without GGOs in the left upper lobe (arrows). The final diagnosis was bronchial pneumonia. b An axial chest CT image of a 37-year-old male without COVID-19 shows centrilobular nodules and tree-in-bud appearance without GGOs in the right middle and left lingula segment (arrows). The final diagnosis was of lung tuberculosis

Fig. 6

Case example defined in the indeterminate for the BSTI guidance statement, and in the typical category for the RSNA expert consensus statement. An axial chest CT image of a 68-year-old male without COVID-19 shows peripheral multifocal GGOs (arrows) in the background of moderate emphysema. The final diagnosis was usual interstitial pneumonia

Fig. 7

a, b ROC curves of both observers to predict COVID-19 according to BSTI and RSNA classification. a For observer A and b for observer B