Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Dec 30;9(1):zrae162.
doi: 10.1093/bjsopen/zrae162.

Inter-rater variability for the American Society of Anesthesiologists classification in patients undergoing hepato-pancreato-biliary surgery (MILESTONE-2): international survey among surgeons and anaesthesiologists

Simone Augustinus  1   2 , Jasper P Sijberden  1   2   3 , Matthanja Bieze  4   5 , Vandana Agarwal  6 , Luca A Aldrighetti  7 , Adnan Alseidi  8 , Francisco C Bonofiglio  9 , Kevin C P Conlon  10 , Katia Donadello  11 , Joris Erdmann  1   2 , Cristina Ferrone  12 , Michael Guertin  13 , Ronald Harter  13 , Maria E Franceschetti  14 , Guiseppe K Fusai  15 , Bas Groot Koerkamp  16 , Thilo Hackert  17 , Jin-Young Jang  18 , Thomas Kander  19 , Tobias Keck  20 , Dominik Krzanicki  21 , Ho-Jin Lee  22 , Keith Lewis  23 , Giuseppe Natalini  14 , Carla Nau  24 , Timothy M Pawlik  25 , Henry A Pitt  26 , Rafaella Reineke  27 , Roberto Salvia  28 , Eduardo de Santibanes  29 , Shailesh V Shrikhande  30 , Martin Smith  31 , Attila Szijarto  32 , Bobby Tingstedt  33 , Alice C Wei  34 , John Windsor  35 , Mohammed Abu Hilal  3 , Manuel Pardo  36 , Markus W Hollmann  4 , Marc G Besselink  1   2
Affiliations

Inter-rater variability for the American Society of Anesthesiologists classification in patients undergoing hepato-pancreato-biliary surgery (MILESTONE-2): international survey among surgeons and anaesthesiologists

Simone Augustinus et al. BJS Open. .

Abstract

Background: Patients undergoing hepato-pancreato-biliary surgery are typically preoperatively assessed using the American Society of Anesthesiologists (ASA) classification, which is also used for case-mix adjustment when comparing centre outcomes. Studies determining the inter-rater variability of the ASA classification within hepato-pancreato-biliary surgery are currently lacking.

Methods: An international survey was collected and a case-vignette study was performed (November 2022-April 2023) regarding the ASA classification in patients undergoing hepato-pancreato-biliary surgery among anaesthesiologists and surgeons from (inter)national societies. The survey consisted of 23 questions and eight case-vignettes. Primary analysis included descriptive statistics and the inter-rater variability was calculated using Light's Kappa.

Results: Overall, 1283 participants from 55 countries responded: 1073 (84%) anaesthesiologists and 210 (16%) surgeons. The ASA classification was commonly used, both clinically 1003/1283 (78%) and for research 728/762 (96%). The majority of respondents (n = 1019, 79%) declared that ASA score impacted their perioperative strategy. There inter-rater variability was fair-moderate (Kappa 0.26-0.42) in all case-vignettes. Inter-rater variability differed within and among geographic regions for each case. Over 80% (n = 1138) of respondents stated that they would take the underlying disease (for example cancer) into account, but this changed the preferred ASA score within the case-vignettes by only 1%. Type of surgery changed the preferred score in the case-vignettes (13% difference). The most common suggestions to improve the ASA classification were to clarify whether type of operation should be considered, create a more extensive definition, and provide more examples.

Conclusions: Inter-rater variability was present within the ASA classification of patients undergoing hepato-pancreato-biliary surgery, which may impact perioperative strategy and hamper research results. Additional guidance to classify patients according to ASA is urgently needed. Until then, more objective measurements should be considered for case-mix adjustment within research.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Quality of American Society of Anesthesiologists (ASA) classification according to respondents Score 1–10 for quality of the ASA classification. 1: I consider ASA a poor classification system, non-objective with very poor interobserver agreement, 10: I consider ASA an excellent classification system, highly objective and excellent interobserver agreement.
Fig. 2
Fig. 2
Does the American Society of Anesthesiologists (ASA) score assigned to a patient change your perioperative strategy?
Fig. 3
Fig. 3
Considerations made when classifying patients according to American Society of Anesthesiologists (ASA) score
Fig. 4
Fig. 4
Considerations made in the American Society of Anesthesiologists (ASA) scoring process Rated on a scale of 1–5, with 1 strongly disagree, and 5 strongly agree.
See this image and copyright information in PMC

References

    1. American Association of Anesthesiologists . ASA Physical Status Classification System 2020. Available from: https://www.asahq.org/standards-and-practice-parameters/statement-on-asa... (accessed 1 October 2024)
    1. De Cassai A, Boscolo A, Tonetti T, Ban I, Ori C. Assignment of ASA-physical status relates to anesthesiologists’ experience: a survey-based national-study. Korean J Anesthesiol 2019;72:53–59 - PMC - PubMed
    1. Haynes SR, Lawler PG. An assessment of the consistency of ASA physical status classification allocation. Anaesthesia 1995;50:195–199 - PubMed
    1. Mak PH, Campbell RC, Irwin MG. The ASA physical status classification: inter-observer consistency. American Society of Anesthesiologists. Anaesth Intensive Care 2002;30:633–640 - PubMed
    1. Ranta S, Hynynen M, Tammisto T. A survey of the ASA physical status classification: significant variation in allocation among Finnish anaesthesiologists. Acta Anaesthesiol Scand 1997;41:629–632 - PubMed

LinkOut - more resources