A Web-Based Tool for Patient Triage in Emergency Department Settings: Validation Using the Emergency Severity Index

Abstract

Background: We evaluated the concordance between triage scores generated by a novel Internet clinical decision support tool, Clinical GPS (cGPS) (Lumiata Inc, San Mateo, CA), and the Emergency Severity Index (ESI), a well-established and clinically validated patient severity scale in use today. Although the ESI and cGPS use different underlying algorithms to calculate patient severity, both utilize a five-point integer scale with level 1 representing the highest severity.

Objective: The objective of this study was to compare cGPS results with an established gold standard in emergency triage.

Methods: We conducted a blinded trial comparing triage scores from the ESI: A Triage Tool for Emergency Department Care, Version 4, Implementation Handbook to those generated by cGPS from the text of 73 sample case vignettes. A weighted, quadratic kappa statistic was used to assess agreement between cGPS derived severity scores and those published in the ESI handbook for all 73 cases. Weighted kappa concordance was defined a priori as almost perfect (kappa > 0.8), substantial (0.6 < kappa < 0.8), moderate (0.4 < kappa < 0.6), fair (0.2 < kappa< 0.4), or slight (kappa < 0.2).

Results: Of the 73 case vignettes, the cGPS severity score matched the ESI handbook score in 95% of cases (69/73 cases), in addition, the weighted, quadratic kappa statistic showed almost perfect agreement (kappa = 0.93, 95% CI 0.854-0.996). In the subanalysis of 41 case vignettes assigned ESI scores of level 1 or 2, the cGPS and ESI severity scores matched in 95% of cases (39/41 cases).

Conclusions: These results indicate that the cGPS is a reliable indicator of triage severity, based on its comparison to a standardized index, the ESI. Future studies are needed to determine whether the cGPS can accurately assess the triage of patients in real clinical environments.

Keywords: clinical decision support; differential diagnosis; emergency severity index; triage.

Figure 1

Overview of the algorithm used to derive the triage score.

Figure 2

The clinical GPS v2.0 (cGPS) Web-based tool takes clinicians through an 8-step process that supports natural language entry (A) and uses autosuggestions and “quick picks” to maximize efficiency (B).

Figure 3

The clinical GPS v2.0 (cGPS) generates differential diagnoses with severity scores and upstream and downstream possibilities, and follow-up questions and tests, including associated costs (C & D).

Figure 4

The clinical GPS v2.0 (cGPS) interfaces directly with the electronic health record (E & F).

Figure 5

Distribution of Emergency Severity Index (ESI) and clinical GPS v2.0 (cGPS) severity scores for the case vignettes (n=73).