Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Observational Study
. 2022 Jan 21;11(1):10.
doi: 10.1186/s13756-022-01050-w.

Reliability and validity of multicentre surveillance of surgical site infections after colorectal surgery

Janneke D M Verberk  1   2   3 , Stephanie M van Rooden  4 , David J Hetem  5 , Herman F Wunderink  6 , Anne L M Vlek  7 , Corianne Meijer  8 , Eva A H van Ravensbergen  9 , Elisabeth G W Huijskens  10 , Saara J Vainio  11 , Marc J M Bonten  6   4 , Maaike S M van Mourik  6
Affiliations
Observational Study

Reliability and validity of multicentre surveillance of surgical site infections after colorectal surgery

Janneke D M Verberk et al. Antimicrob Resist Infect Control. .

Abstract

Background: Surveillance is the cornerstone of surgical site infection prevention programs. The validity of the data collection and awareness of vulnerability to inter-rater variation is crucial for correct interpretation and use of surveillance data. The aim of this study was to investigate the reliability and validity of surgical site infection (SSI) surveillance after colorectal surgery in the Netherlands.

Methods: In this multicentre prospective observational study, seven Dutch hospitals performed SSI surveillance after colorectal surgeries performed in 2018 and/or 2019. When executing the surveillance, a local case assessment was performed to calculate the overall percentage agreement between raters within hospitals. Additionally, two case-vignette assessments were performed to estimate intra-rater and inter-rater reliability by calculating a weighted Cohen's Kappa and Fleiss' Kappa coefficient. To estimate the validity, answers of the two case-vignettes questionnaires were compared with the answers of an external medical panel.

Results: 1111 colorectal surgeries were included in this study with an overall SSI incidence of 8.8% (n = 98). From the local case assessment it was estimated that the overall percent agreement between raters within a hospital was good (mean 95%, range 90-100%). The Cohen's Kappa estimated for the intra-rater reliability of case-vignette review varied from 0.73 to 1.00, indicating substantial to perfect agreement. The inter-rater reliability within hospitals showed more variation, with Kappa estimates ranging between 0.61 and 0.94. In total, 87.9% of the answers given by the raters were in accordance with the medical panel.

Conclusions: This study showed that raters were consistent in their SSI-ascertainment (good reliability), but improvements can be made regarding the accuracy (moderate validity). Accuracy of surveillance may be improved by providing regular training, adapting definitions to reduce subjectivity, and by supporting surveillance through automation.

Keywords: Colorectal surgery; Epidemiology; Infection prevention; Inter-rater reliability; Surgical site infection; Surveillance.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Overview of SSI incidence per hospital accounting for the number of surgical procedures. The black dotted line shows the mean incidence rate, the grey curved lines are the corresponding 95% confidence interval. a Overview of all SSIs per hospital. b Overview of superficial SSIs per hospital. c Overview of deep SSIs per hospital
See this image and copyright information in PMC

References

    1. Magill SS, Edwards JR, Bamberg W, Beldavs ZG, Dumyati G, Kainer MA, et al. Multistate point-prevalence survey of health care-associated infections. N Engl J Med. 2014;370(13):1198–1208. doi: 10.1056/NEJMoa1306801. - DOI - PMC - PubMed
    1. Koek MBG, van der Kooi TII, Stigter FCA, de Boer PT, de Gier B, Hopmans TEM, et al. Burden of surgical site infections in the Netherlands: cost analyses and disability-adjusted life years. J Hosp Infect. 2019;103(3):293–302. doi: 10.1016/j.jhin.2019.07.010. - DOI - PubMed
    1. Kirkland KB, Briggs JP, Trivette SL, Wilkinson WE, Sexton DJ. The impact of surgical-site infections in the 1990s: attributable mortality, excess length of hospitalization, and extra costs. Infect Control Hosp Epidemiol. 1999;20(11):725–730. doi: 10.1086/501572. - DOI - PubMed
    1. Tanner J, Khan D, Aplin C, Ball J, Thomas M, Bankart J. Post-discharge surveillance to identify colorectal surgical site infection rates and related costs. J Hosp Infect. 2009;72(3):243–250. doi: 10.1016/j.jhin.2009.03.021. - DOI - PubMed
    1. Shaw E, Gomila A, Piriz M, Perez R, Cuquet J, Vazquez A, et al. Multistate modelling to estimate excess length of stay and risk of death associated with organ/space infection after elective colorectal surgery. J Hosp Infect. 2018;100(4):400–405. doi: 10.1016/j.jhin.2018.08.010. - DOI - PubMed

Publication types

LinkOut - more resources