Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Oct 1;111(10):znae253.
doi: 10.1093/bjs/znae253.

Development and validation of a novel tool for identification and categorization of non-technical errors associated with surgical mortality

Jesse D Ey  1 , Victoria Kollias  1 , Matheesha B Herath  1 , Octavia Lee  1 , Martin H Bruening  1 , Adam J Wells  2 , Guy J Maddern  1
Affiliations

Development and validation of a novel tool for identification and categorization of non-technical errors associated with surgical mortality

Jesse D Ey et al. Br J Surg. .

Abstract

Background: Up to half of all surgical adverse events are due to non-technical errors, making non-technical skill assessment and improvement a priority. No specific tools are available to retrospectively identify non-technical errors that have occurred in surgical patient care. This original study aimed to develop and provide evidence of validity and inter-rater reliability for the System for Identification and Categorization of Non-technical Error in Surgical Settings (SICNESS).

Methods: A literature review, modified Delphi process, and two pilot phases were used to develop and test the SICNESS tool. For each pilot, 12 months of surgical mortality data from the Australian and New Zealand Audit of Surgical Mortality were assessed by two independent reviewers using the SICNESS tool. Main outcomes included tool validation through modified Delphi consensus, and inter-rater reliability for: non-technical error identification and non-technical error categorization using Cohen's κ coefficient, and overall agreement using Fleiss' κ coefficient.

Results: Version 1 of the SICNESS was used for pilot 1, including 412 mortality cases, and identified and categorized non-technical errors with strong-moderate inter-rater reliability. Non-technical error exemplars were created and validated through Delphi consensus, and a novel mental model was developed. Pilot 2 included an additional 432 mortality cases. Inter-rater reliability was near perfect for leadership (κ 0.92, 95% c.i. 0.82 to 1.00); strong for non-technical error identification (κ 0.89, 0.84 to 0.93), communication and teamwork (κ 0.89, 0.79 to 0.99), and decision-making (κ 0.85, 0.79 to 0.92); and moderate for situational awareness (κ 0.79, 0.71 to 0.87) and overall agreement (κ 0.69, 0.66 to 0.73).

Conclusion: The SICNESS is a reliable and valid tool, enabling retrospective identification and categorization of non-technical errors associated with death, occurring in real surgical patient interactions.

Plain language summary

Many errors in surgery occur because of poor non-technical skills. The aim of this study was to create a tool to identify this type of error using patient data so future errors may be prevented. The tool was designed through expert opinion and literature review. It was tested using surgical patient death data from Australia and New Zealand. The final tool was able to identify and group non-technical errors reliably. This tool makes it possible to identify non-technical errors so future errors may be reduced.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Consensus agreed domains and elements of SICNESS SICNESS, System for Identification and Categorization of Non-technical Error in Surgical Settings.
Fig. 2
Fig. 2
Novel mental model: primary versus resultant non-technical errors SICNESS, System for Identification and Categorization of Non-technical Error in Surgical Settings.
See this image and copyright information in PMC

References

    1. Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 2008;32:548–556 - PubMed
    1. WHO . WHO Guidelines for Safe Surgery: Safe Surgery Saves Lives. https://iris.who.int/bitstream/handle/10665/44185/9789241598552_eng.pdf?... (accessed 19 July 2024)
    1. Murshed I, Gupta AK, Camilos AN, Sabab A, Bacchi S, Kovoor JG et al. Surgical interhospital transfer mortality: national analysis. Br J Surg 2023;110:591–598 - PMC - PubMed
    1. Gupta AK, Stewart SK, Cottell K, McCulloch GA, Babidge W, Maddern GJ. Potentially avoidable issues in neurosurgical mortality cases in Australia: identification and improvements. ANZ J Surg 2017;87:86–91 - PubMed
    1. Gupta AK, Stewart SK, Cottell K, McCulloch GA, Miller J, Babidge WJ et al. Potentially avoidable issues in urology mortality cases in Australia: identification and improvements. ANZ J Surg 2020;90:719–724 - PubMed

Publication types