Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 9;7(1):87.
doi: 10.1038/s41746-024-01061-4.

The algorithm journey map: a tangible approach to implementing AI solutions in healthcare

William Boag #  1 , Alifia Hasan #  1 , Jee Young Kim  1 , Mike Revoir  1 , Marshall Nichols  1 , William Ratliff  1 , Michael Gao  1 , Shira Zilberstein  1   2 , Zainab Samad  3 , Zahra Hoodbhoy  3 , Mushyada Ali  3 , Nida Saddaf Khan  3 , Manesh Patel  4 , Suresh Balu  1 , Mark Sendak  5
Affiliations

The algorithm journey map: a tangible approach to implementing AI solutions in healthcare

William Boag et al. NPJ Digit Med. .

Abstract

When integrating AI tools in healthcare settings, complex interactions between technologies and primary users are not always fully understood or visible. This deficient and ambiguous understanding hampers attempts by healthcare organizations to adopt AI/ML, and it also creates new challenges for researchers to identify opportunities for simplifying adoption and developing best practices for the use of AI-based solutions. Our study fills this gap by documenting the process of designing, building, and maintaining an AI solution called SepsisWatch at Duke University Health System. We conducted 20 interviews with the team of engineers and scientists that led the multi-year effort to build the tool, integrate it into practice, and maintain the solution. This "Algorithm Journey Map" enumerates all social and technical activities throughout the AI solution's procurement, development, integration, and full lifecycle management. In addition to mapping the "who?" and "what?" of the adoption of the AI tool, we also show several 'lessons learned' throughout the algorithm journey maps including modeling assumptions, stakeholder inclusion, and organizational structure. In doing so, we identify generalizable insights about how to recognize and navigate barriers to AI/ML adoption in healthcare settings. We expect that this effort will further the development of best practices for operationalizing and sustaining ethical principles-in algorithmic systems.

PubMed Disclaimer

Conflict of interest statement

M.S., S.B., W.R., M.N., M.R. and M.G. reported co-inventing software at Duke University licensed by Duke University to external commercial entities including Clinetic, Cohere Med, Kela Health, and Fullsteam Health. M.S., S.B., M.N., M.R. and M.G. own equity in Clinetic. No other disclosures were reported.

Figures

Fig. 1
Fig. 1
Symbols used in the algorithm journey map.
Fig. 2
Fig. 2
Journey map of problem identification phase.
Fig. 3
Fig. 3
Journey map of the development phase.
Fig. 4
Fig. 4
Journey map of model build and validation sub-phase of development.
Fig. 5
Fig. 5
Journey map of the development of user interface and user experience.
Fig. 6
Fig. 6
Journey map of the integration phase.
Fig. 7
Fig. 7
Journey map of technical integration sub-phase.
Fig. 8
Fig. 8
Journey map of clinical integration sub-phase.
Fig. 9
Fig. 9
Different tasks that arise throughout post-rollout lifecycle management.
See this image and copyright information in PMC

References

    1. Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc. J. 2019;6:94–98. doi: 10.7861/futurehosp.6-2-94. - DOI - PMC - PubMed
    1. Park E. The AI Bill of Rights: a step in the right direction. Orange Cty. Lawyer Mag. 2023;65:2.
    1. Boag, W. Evidence-based AI Ethics. Dissertation, Massachusetts Institute of Technology (2022).
    1. Zhou Q, Chen Z-h, Cao Y-h, Peng S. Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review. NPJ Digit. Med. 2021;4:154. doi: 10.1038/s41746-021-00524-2. - DOI - PMC - PubMed
    1. Chen P-HC, Liu Y, Peng L. How to develop machine learning models for healthcare. Nat. Mater. 2019;18:410–414. doi: 10.1038/s41563-019-0345-0. - DOI - PubMed