The impact of COVID-19 on a high-volume incident learning system: A retrospective analysis

Abstract

Purpose: The purpose of this work was to assess how the coronavirus disease 2019 (COVID-19) pandemic impacted our incident learning system data and communicate the impact of a major exogenous event on radiation oncology clinical practice.

Methods: Trends in our electronic incident reporting system were analyzed to ascertain the impact of the COVID-19 pandemic, including any direct clinical changes. Incident reports submitted in the 18 months prior to the pandemic (September 14, 2018 to March 13, 2020) and reports submitted during the first 18 months of the pandemic (March 14, 2020 to September 13, 2021) were compared. The incident reports include several data elements that were evaluated for trends between the two time periods, and statistical analysis was performed to compare the proportions of reports.

Results: In the 18 months prior to COVID-19, 192 reports were submitted per 1000 planning tasks (n = 832 total). In the first 18 months of the pandemic, 147 reports per 1000 planning tasks were submitted (n = 601 total), a decrease of 23.4%. Statistical analysis revealed that there were no significant changes among the data elements between the pre- and during COVID-19 time periods. An analysis of the free-text narratives in the reports found that phrases related to pretreatment imaging were common before COVID-19 but not during. Conversely, phrases related to intravenous contrast, consent for computed tomography, and adaptive radiotherapy became common during COVID-19.

Conclusions: The data elements captured by our incident learning system were stable after the onset of the COVID-19 pandemic, with no statistically significant findings after correction for multiple comparisons. A trend toward fewer reports submitted for low-risk issues was observed. The methods used in the work can be generalized to events with a large-scale impact on the clinic or to monitor an incident learning system to drive future improvement activities.

Keywords: COVID-19; Incident learning; Radiotherapy.

FIGURE 1

The number of incident reports submitted (blue/orange bars) and the number of planning tasks (green line) per month. The average number of incident reports submitted in the 18 months before coronavirus disease 2019 (COVID‐19) was 46.2 per month versus 33.4 per month in the 18 months during COVID‐19. The transition date between pre‐ and during COVID‐19 occurred in the middle of March 2020, resulting in half‐months of data at the edges of each subplot. The gray rectangles for these months represent double the number of reports during the half‐month, which gives a better sense of the trend over time

FIGURE 2

The number of reports submitted per 1000 planning tasks categorized by where in the radiotherapy process the issue was discovered. Although not statistically significant, we observed a large increase during coronavirus disease 2019 (COVID‐19) for events discovered before simulation and a large decrease in events discovered at simulation and during pretreatment quality assurance (QA)

FIGURE 3

Reports per 1000 planning tasks categorized by the role of the reporter. NP, nurse practitioner; PA, physician assistant

FIGURE 4

Reports per 1000 planning tasks categorized by event classification

FIGURE 5

Reports per 1000 planning tasks categorized by their potential severity

FIGURE 6

The types of interventions implemented in response to incident reports as a percentage of total reports

FIGURE 7

Word clouds showing the frequency of two‐ and three‐word phrases included in the free‐text narrative submitted with the incident report