The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery

Abstract

Background: In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR).

Methods: Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated.

Results: From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5-99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm.

Conclusions: The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4-12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources.

Keywords: Algorithm; Automated surveillance; Colorectal surgery; Healthcare-associated infections; Natural language processing; Surgical site infections.

Fig. 1

Flow chart of the study ECDC: European Centre for Disease Prevention and Control; NLP: natural language processing; SSI: surgical site infection. Semi-automated algorithm as published in Verberk et al. [18].

Fig. 2

Flow diagram of natural language processing-augmented surveillance algorithm for deep surgical site infections NLP: natural language processing; SSI: surgical site infection. Schematic overview of the original semi-automated algorithm comprised of structured data (grey frame), augmented with unstructured data from clinical notes (blue frame). Admissions: Length of stay of index admission ≥ 14 days or 1 readmission to original department or in-hospital mortality within follow-up (FU) time (= 45 days after surgery). Re-surgery: ≥1 reoperation by original surgery specialty after the index surgery but within FU time. Radiology: ≥1 orders for CT scan within FU time. Antibiotics: ≥3 consecutive days of antibiotics (ATC J01) within FU time, starting from day 2 (index surgery = day 0). Fulfilling 2–4 of the above components classifies surgery as high probability for deep SSI.

Fig. 3

Rule-based component

Fig. 4

Heat map for the distribution of keywords among patients with and without deep SSI * Proximity search, keyword must be within a distance of five words from one of the following locations: incision, operation wound, abdomen, wound, pelvis, duodenum, flank, gall bladder, skin, intra-abdominal, next to anastomosis, colon, liver, pancreas, abdomen/stomach, between small intestines, operation area, operation wound, presacral, rectum, retroperitoneal, incision, intestine, small intestine, under diaphragm, midline incision, midline wound, sutures / stitches / (surgical) staples, stomach. # The following antibiotics including their brand names: piperacillin-tazobactam, meropenem, imipenem, metronidazole, ciprofloxacin, cefotaxime, trimethoprim-sulfa, cefuroxime, amoxicillin.