Automated tracking of emergency department abdominal CT findings during the COVID-19 pandemic using natural language processing

Abstract

Purpose: During the COVID-19 pandemic, emergency department (ED) volumes have fluctuated. We hypothesized that natural language processing (NLP) models could quantify changes in detection of acute abdominal pathology (acute appendicitis (AA), acute diverticulitis (AD), or bowel obstruction (BO)) on CT reports.

Methods: This retrospective study included 22,182 radiology reports from CT abdomen/pelvis studies performed at an urban ED between January 1, 2018 to August 14, 2020. Using a subset of 2448 manually annotated reports, we trained random forest NLP models to classify the presence of AA, AD, and BO in report impressions. Performance was assessed using 5-fold cross validation. The NLP classifiers were then applied to all reports.

Results: The NLP classifiers for AA, AD, and BO demonstrated cross-validation classification accuracies between 0.97 and 0.99 and F1-scores between 0.86 and 0.91. When applied to all CT reports, the estimated numbers of AA, AD, and BO cases decreased 43-57% in April 2020 (first regional peak of COVID-19 cases) compared to 2018-2019. However, the number of abdominal pathologies detected rebounded in May-July 2020, with increases above historical averages for AD. The proportions of CT studies with these pathologies did not significantly increase during the pandemic period.

Conclusion: Dramatic decreases in numbers of acute abdominal pathologies detected by ED CT studies were observed early on during the COVID-19 pandemic, though these numbers rapidly rebounded. The proportions of CT cases with these pathologies did not increase, which suggests patients deferred care during the first pandemic peak. NLP can help automatically track findings in ED radiology reporting.

Keywords: Appendicitis; Bowel obstruction; COVID-19; CT; Diverticulitis; Emergency.

Fig. 1

Schematic of the study design. This analysis was repeated for 3 acute abdominal pathologies-of-interest including acute appendicitis, acute diverticulitis, and bowel obstruction. ED, emergency department; NLP, natural language processing.

Fig. 2

Trends in acute abdominal pathology detected on CT over time from January 2018 to July 2020. (A) Line plot for the number of ED CT abdomen/pelvis studies performed by month. (B) Line plot for the estimated number of cases of acute appendicitis, acute diverticulitis, and bowel obstruction detected by NLP analysis of radiology report impressions by month. (C) Line plot for the estimated proportion of CT studies performed with acute abdominal pathology detected by NLP analysis (case positivity rate) by month. The same figure legend for plots (B) and (C) is shown below plot (C).

Fig. 3

Trends in acute abdominal pathology detected by CT in 2020, represented as the proportional change by month relative to the average over 24 months from 2018 to 2019. (A) Line plot for the proportional change in ED CT abdomen/pelvis studies and estimated number of cases of acute appendicitis, acute diverticulitis and bowel obstruction relative to the average from 2018 to 2019. (B) Line plot for the proportional change in the estimated proportion of total ED CT abdomen/pelvis studies performed with acute appendicitis, acute diverticulitis or bowel obstruction detected by NLP analysis relative to the average from 2018 to 2019.