Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Observational Study
. 2019 Nov;47(11):1477-1484.
doi: 10.1097/CCM.0000000000003803.

Clinician Perception of a Machine Learning-Based Early Warning System Designed to Predict Severe Sepsis and Septic Shock

Jennifer C Ginestra  1 , Heather M Giannini  1 , William D Schweickert  2   3 , Laurie Meadows  4 , Michael J Lynch  4 , Kimberly Pavan  5 , Corey J Chivers  3 , Michael Draugelis  3 , Patrick J Donnelly  6 , Barry D Fuchs  2   3 , Craig A Umscheid  3   7   8
Affiliations
Observational Study

Clinician Perception of a Machine Learning-Based Early Warning System Designed to Predict Severe Sepsis and Septic Shock

Jennifer C Ginestra et al. Crit Care Med. 2019 Nov.

Abstract

Objective: To assess clinician perceptions of a machine learning-based early warning system to predict severe sepsis and septic shock (Early Warning System 2.0).

Design: Prospective observational study.

Setting: Tertiary teaching hospital in Philadelphia, PA.

Patients: Non-ICU admissions November-December 2016.

Interventions: During a 6-week study period conducted 5 months after Early Warning System 2.0 alert implementation, nurses and providers were surveyed twice about their perceptions of the alert's helpfulness and impact on care, first within 6 hours of the alert, and again 48 hours after the alert.

Measurements and main results: For the 362 alerts triggered, 180 nurses (50% response rate) and 107 providers (30% response rate) completed the first survey. Of these, 43 nurses (24% response rate) and 44 providers (41% response rate) completed the second survey. Few (24% nurses, 13% providers) identified new clinical findings after responding to the alert. Perceptions of the presence of sepsis at the time of alert were discrepant between nurses (13%) and providers (40%). The majority of clinicians reported no change in perception of the patient's risk for sepsis (55% nurses, 62% providers). A third of nurses (30%) but few providers (9%) reported the alert changed management. Almost half of nurses (42%) but less than a fifth of providers (16%) found the alert helpful at 6 hours.

Conclusions: In general, clinical perceptions of Early Warning System 2.0 were poor. Nurses and providers differed in their perceptions of sepsis and alert benefits. These findings highlight the challenges of achieving acceptance of predictive and machine learning-based sepsis alerts.

PubMed Disclaimer

Figures

Figure 1:
Figure 1:. Clinician Perceived Utility of Alert.
Unhelpful: Combined percentage of those responding, “Very unhelpful” and “Unhelpful” on a Likert Scale. Neither: Percentage of those responding “Neither helpful nor unhelpful” on a Likert Scale. Helpful: Combined percentage of those responding “Very helpful” and “Helpful” on a Likert Scale. At Time of Alert: First Survey responses, submitted within 6 hours following the alert. 48 Hours After Alert: Second Survey responses, submitted at least 48 hours after alert.
Figure 2:
Figure 2:. Clinician Perceived Impact on Patient Care.
Improved Care: Combined percentage of those responding, “Definitely improved care” and “Probably improved care” on a Likert Scale. Maybe Improved Care: Percentage of those responding “Maybe improved care” on a Likert Scale. Didn’t Improve Care: Combined percentage of those responding, “Definitely did not improve care” and “Probably did not improve care” on a Likert Scale. At Time of Alert: First Survey responses, submitted within 6 hours following the alert. 48 Hours After Alert: Second Survey responses, submitted at least 48 hours after alert.
See this image and copyright information in PMC

Comment in

References

    1. Rhee C, Dantes R, Epstein L, et al.: Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009–2014. JAMA 2017; 318(13):1241–1249 - PMC - PubMed
    1. Jones SL, Ashton CM, Kiehne LB, et al.: Outcomes and resource use of sepsis-associated stays by presence on admission, severity, and hospital type. Medical Care 2016; 54(3):303–310 - PMC - PubMed
    1. Levy MM, Rhodes A, Phillips GS, et al.: Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study. Intensive Care Med 2014; 40(11):1623–1633 - PubMed
    1. Kumar A, Roberts D, Wood KE, et al.: Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006; 34(6):1589–1596 - PubMed
    1. Bhattacharjee P, Edelson DP, Churpek MM: Identifying patients with sepsis on the hospital wards. Chest 2017; 151(4):898–907 - PMC - PubMed

Publication types