Accuracy of the modified Global Burden of Disease International Classification of Diseases coding methods for identifying sepsis: a prospective multicentre cohort study

Abstract

Background: This study assessed the accuracy of three International Classification of Diseases (ICD) codes methods derived from Global Burden of Disease (GBD) sepsis study (modified GBD method) in identifying sepsis, compared to the Angus method. Sources of errors in these methods were also reported.

Methods: Prospective multicentre, observational, study. Emergency Department patients aged ≥ 16 years with high sepsis risk from nine hospitals in NSW, Australia were screened for clinical sepsis using Sepsis 3 criteria and coded as having sepsis or not using the modified GBD and Angus methods. The three modified GBD methods were: Explicit-sepsis-specific ICD code recorded; Implicit-sepsis-specific code or infection as primary ICD code plus organ dysfunction code; Implicit plus-as for Implicit but infection as primary or secondary ICD code. Agreement between clinical sepsis and ICD coding methods was assessed using Cronbach alpha (α). For false positive cases (ICD-coded sepsis but not clinically diagnosed), the ICD codes leading to those errors were documented. For false negatives (clinically diagnosed sepsis but ICD-coded), uncoded sources of infection and organ dysfunction were documented.

Results: Of 6869 screened patients, 450 (median age 72.4 years, 48.9% females) met inclusion criteria. Clinical sepsis was diagnosed in 215/450 (47.8%). The explicit, implicit, implicit plus and Angus methods identified sepsis in 108/450 (24.0%), 175/450 (38.9%), 222/450 (49.3%) and 170/450 (37.8%), respectively. Sensitivity was 41.4%, 58.1%, 67.4% and 55.8%, and specificity 91.9%, 78.7%, 67.2% and 79.1%, respectively. Agreement between clinical sepsis and all ICD coding methods was low (α = 0.52-0.56). False positives were 19, 50, and 77, while false negatives were 126, 90, and 70 for the explicit, implicit, and implicit plus methods, respectively. For false positive cases, unspecified urinary tract infection, hypotension and acute kidney failure were commonly assigned infection and organ dysfunction codes. About half (44.3%-55.6%) of the false negative cases didn't have a pathogen documented.

Conclusion: The modified GBD method demonstrated low accuracy in identifying sepsis; with the implicit plus method being the most accurate. Errors in identifying sepsis using ICD codes arise mostly from coding for unspecified urinary infections and associated organ dysfunction.

Trial registration: The study was registered at the ANZCTR (ACTRN12621000333819) on 24 March 2021.

Keywords: Diagnostic accuracy; ICD code; Sensitivity; Sepsis; Specificity.

Fig. 1

Study flow. Clinical sepsis: Number of patients who met Sepsis-3 criteria. *Presence of one of the explicit sepsis ICD-10-AM codes (Additional File 1; Supplementary Table 1) as the primary or secondary diagnosis. **Presence of an infection code listed as the primary diagnosis and an “organ dysfunction code” listed as secondary diagnosis from the modified GBD codes (Additional File 1; Supplementary Tables 2 and 3) OR one of the explicit sepsis codes (Additional File 1; Supplementary Table 1). ^$Presence of an infection code and an “organ dysfunction code” from the modified GBD codes (Additional File 1; Supplementary Tables 2 and 3) OR one of the explicit sepsis codes (Additional File 1; Supplementary Table 1). ***Presence of an infection code and an “organ dysfunction code” from the Angus codes (Additional File 1; Supplementary Tables 2 and 3) or R57.2 or R65.1 from the explicit sepsis codes (Additional File 1; Supplementary Table 1). GBD, Global Burden of Disease; ICD-10-AM, International Classification of Diseases-10th Revision-Australian Modification. Note: The number of patients in various groups are not mutually exclusive

Fig. 2

Agreement between clinical sepsis and various ICD coding methods. Note: Size of circle is proportional to number of patients in a group; overlapped area indicates degree of agreement. Numbers in the overlapping areas indicate the number of patients satisfying multiple criteria. Clinical sepsis: Number of patients who met Sepsis-3 criteria. Explicit: Presence of one of the explicit sepsis ICD-10-AM codes (Additional File 1; Supplementary Table 1) as the primary or secondary diagnosis. Implicit: Presence of an infection code listed as the primary diagnosis and an “organ dysfunction code” listed as secondary diagnosis from the modified GBD codes (Additional File 1; Supplementary Tables 2 and 3) OR one of the explicit sepsis codes (Additional File 1; Supplementary Table 1). Implicit plus: Presence of an infection code and an “organ dysfunction code” from the modified GBD codes (Additional File 1; Supplementary Tables 2 and 3) OR one of the explicit sepsis codes (Additional File 1; Supplementary Table 1). Angus: Presence of an infection code and an “organ dysfunction code” from the Angus codes (Additional File 1; Supplementary Tables 2 and 3) or R57.2 or R65.1 from the explicit sepsis codes (Additional File 1; Supplementary Table 1). GBD, Global Burden of Disease; ICD-10-AM, International Classification of Diseases-10th Revision- Australian Modification

Fig. 3

Heat map of combinations of infection codes and organ dysfunction ICD-10-AM codes in false positive cases. A Implicit modified GBD method (N = 50). Note: More than one infection and/or organ dysfunction code was present per patient. ICD-10-AM, international classification of disease-10th revision-Australian modification; GBD, Global Burden of Disease. Implicit: Presence of an infection code listed as the primary diagnosis and an “organ dysfunction code” listed as secondary diagnosis from the modified GBD codes (Additional File 1, Supplementary Tables 2 and 3) OR one of the explicit sepsis codes (Additional File 1; Supplementary Table 1). B Implicit plus modified GBD method (N = 77). Note: More than one infection and/or organ dysfunction code was present per patient. ICD-10-AM: International Classification of Disease-10th Revision- Australian Modification; GBD: Global Burden of Disease. Implicit plus: Presence of an infection code and an “organ dysfunction code” from the modified GBD codes (Additional File 1; Supplementary Tables 2 and 3) OR one of the explicit sepsis codes (Additional File 1; Supplementary Table 1)

Fig. 4

Distribution of infection and organ dysfunction ICD-10-AM codes in false negative cases in implicit and implicit plus modified GBD methods. ICD-10-AM, international classification of disease; OD, organ dysfunction; GBD, Global Burden of Disease. Implicit: Presence of an infection code listed as the primary diagnosis and an “organ dysfunction code” listed as secondary diagnosis from the modified GBD codes OR one of the explicit sepsis codes. Implicit plus: Presence of an infection code and an “organ dysfunction code” from the modified GBD codes (Additional File 1; Supplementary Tables 2 and 3) OR one of the explicit sepsis codes (Additional File 1; Supplementary Table 1)