Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Apr-Jun;47(2):E21-E31.
doi: 10.1097/HMR.0000000000000324.

Enhancing the value to users of machine learning-based clinical decision support tools: A framework for iterative, collaborative development and implementation

Sara J Singer, Katherine C Kellogg, Ari B Galper, Deborah Viola

Enhancing the value to users of machine learning-based clinical decision support tools: A framework for iterative, collaborative development and implementation

Sara J Singer et al. Health Care Manage Rev. 2022 Apr-Jun.

Abstract

Background: Health care organizations are integrating a variety of machine learning (ML)-based clinical decision support (CDS) tools into their operations, but practitioners lack clear guidance regarding how to implement these tools so that they assist end users in their work.

Purpose: We designed this study to identify how health care organizations can facilitate collaborative development of ML-based CDS tools to enhance their value for health care delivery in real-world settings.

Methodology/approach: We utilized qualitative methods, including 37 interviews in a large, multispecialty health system that developed and implemented two operational ML-based CDS tools in two of its hospital sites. We performed thematic analyses to inform presentation of an explanatory framework and recommendations.

Results: We found that ML-based CDS tool development and implementation into clinical workflows proceeded in four phases: iterative solution coidentification, iterative coengagement, iterative coapplication, and iterative corefinement. Each phase is characterized by a collaborative back-and-forth process between the technology's developers and users, through which both users' activities and the technology itself are transformed.

Conclusion: Health care organizations that anticipate iterative collaboration to be an integral aspect of their ML-based CDS tools' development and implementation process may have more success in deploying ML-based CDS tools that assist end users in their work than organizations that expect a traditional technology innovation process.

Practice implications: Managers developing and implementing ML-based CDS tools should frame the work as a collaborative learning opportunity for both users and the technology itself and should solicit constructive feedback from users on potential changes to the technology, in addition to potential changes to user workflows, in an ongoing, iterative manner.

PubMed Disclaimer

References

    1. Amarasingham R., Moore B. J., Tabak Y. P., Drazner M. H., Clark C. A., Zhang S., Reed W. G., Swanson T. S., Ma Y., Halm E. A. (2010). An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Medical Care, 48, 981–988.
    1. Angelo S. A., Arruda E. F., Goldwasser R., Lobo M. S., Salles A. (2017). Demand forecast and optimal planning of intensive care unit (ICU) capacity. Pesquisa Operacional, 37(2), 229–245.
    1. Barnes S., Hamrock E., Toerper M., Siddiqui S., Levin S. (2016). Real-time prediction of inpatient length of stay for discharge prioritization. Journal of the American Medical Informatics Association, 23(e1), e2–e10.
    1. Barrett M., Oborn E., Orlikowski W. J., Yates J. (2012). Reconfiguring boundary relations: Robotic innovations in pharmacy work. Organization Science, 23(5), 1448–1466.
    1. Becker H. S. (1998). Tricks of the trade: How to think about your research while you're doing it. University of Chicago Press.