Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Jul 15;8(7):e15182.
doi: 10.2196/15182.

Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study

Mark P Sendak  1 , William Ratliff  1 , Dina Sarro  2 , Elizabeth Alderton  2 , Joseph Futoma  3   4 , Michael Gao  1 , Marshall Nichols  1 , Mike Revoir  1 , Faraz Yashar  3 , Corinne Miller  2 , Kelly Kester  2 , Sahil Sandhu  5 , Kristin Corey  1   6 , Nathan Brajer  1   6 , Christelle Tan  1   6 , Anthony Lin  1   6 , Tres Brown  7 , Susan Engelbosch  7 , Kevin Anstrom  8 , Madeleine Clare Elish  9 , Katherine Heller  3   10 , Rebecca Donohoe  11 , Jason Theiling  11 , Eric Poon  7   12 , Suresh Balu  1   6 , Armando Bedoya  7   13 , Cara O'Brien  12
Affiliations

Real-World Integration of a Sepsis Deep Learning Technology Into Routine Clinical Care: Implementation Study

Mark P Sendak et al. JMIR Med Inform. .

Abstract

Background: Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature.

Objective: This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care.

Methods: In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch.

Results: Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch.

Conclusions: Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.

Keywords: change; deep learning; innovation, organizational; machine learning; sepsis; translational medicine.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: MS, WR, JF, MG, MN, MR, NB, AL, KH, AB, and CO are named inventors of the Sepsis Watch deep learning model, which was licensed from Duke University by Cohere Med, Inc. These authors do not hold any equity in Cohere Med, Inc. No other authors have relevant financial disclosures.

Figures

Figure 1
Figure 1
Sepsis Watch swimlane diagram.
Figure 2
Figure 2
Sepsis Watch user interface.
Figure 3
Figure 3
Sepsis Watch training one-page overview.
Figure 4
Figure 4
Timeline of steps involved in translation of Sepsis Watch from problem identification to integration into routine clinical care.
See this image and copyright information in PMC

References

    1. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019 Jan;25(1):44–56. doi: 10.1038/s41591-018-0300-7. - DOI - PubMed
    1. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DG, PROGRESS Group Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. doi: 10.1371/journal.pmed.1001381. http://dx.plos.org/10.1371/journal.pmed.1001381 - DOI - DOI - PMC - PubMed
    1. Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017;4(1):e000234. doi: 10.1136/bmjresp-2017-000234. https://bmjopenrespres.bmj.com/cgi/pmidlookup?view=long&pmid=29435343 - DOI - PMC - PubMed
    1. Abràmoff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med. 2018;1:39. doi: 10.1038/s41746-018-0040-6. http://europepmc.org/abstract/MED/31304320 - DOI - PMC - PubMed
    1. Sendak M, Gao M, Nichols M, Lin A, Balu S. Machine learning in health care: a critical appraisal of challenges and opportunities. EGEMS (Wash DC) 2019 Jan 24;7(1):1. doi: 10.5334/egems.287. http://europepmc.org/abstract/MED/30705919 - DOI - PMC - PubMed

LinkOut - more resources