Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 May 12;9(2):e33960.
doi: 10.2196/33960.

Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study

Jessica M Schwartz  1   2 , Maureen George  2 , Sarah Collins Rossetti  1   2 , Patricia C Dykes  3   4 , Simon R Minshall  5 , Eugene Lucas  1   6 , Kenrick D Cato  2   7
Affiliations

Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study

Jessica M Schwartz et al. JMIR Hum Factors. .

Abstract

Background: Clinician trust in machine learning-based clinical decision support systems (CDSSs) for predicting in-hospital deterioration (a type of predictive CDSS) is essential for adoption. Evidence shows that clinician trust in predictive CDSSs is influenced by perceived understandability and perceived accuracy.

Objective: The aim of this study was to explore the phenomenon of clinician trust in predictive CDSSs for in-hospital deterioration by confirming and characterizing factors known to influence trust (understandability and accuracy), uncovering and describing other influencing factors, and comparing nurses' and prescribing providers' trust in predictive CDSSs.

Methods: We followed a qualitative descriptive methodology conducting directed deductive and inductive content analysis of interview data. Directed deductive analyses were guided by the human-computer trust conceptual framework. Semistructured interviews were conducted with nurses and prescribing providers (physicians, physician assistants, or nurse practitioners) working with a predictive CDSS at 2 hospitals in Mass General Brigham.

Results: A total of 17 clinicians were interviewed. Concepts from the human-computer trust conceptual framework-perceived understandability and perceived technical competence (ie, perceived accuracy)-were found to influence clinician trust in predictive CDSSs for in-hospital deterioration. The concordance between clinicians' impressions of patients' clinical status and system predictions influenced clinicians' perceptions of system accuracy. Understandability was influenced by system explanations, both global and local, as well as training. In total, 3 additional themes emerged from the inductive analysis. The first, perceived actionability, captured the variation in clinicians' desires for predictive CDSSs to recommend a discrete action. The second, evidence, described the importance of both macro- (scientific) and micro- (anecdotal) evidence for fostering trust. The final theme, equitability, described fairness in system predictions. The findings were largely similar between nurses and prescribing providers.

Conclusions: Although there is a perceived trade-off between machine learning-based CDSS accuracy and understandability, our findings confirm that both are important for fostering clinician trust in predictive CDSSs for in-hospital deterioration. We found that reliance on the predictive CDSS in the clinical workflow may influence clinicians' requirements for trust. Future research should explore the impact of reliance, the optimal explanation design for enhancing understandability, and the role of perceived actionability in driving trust.

Keywords: clinical decision support systems; inpatient; machine learning; nurses; physicians; qualitative research.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Human-computer trust conceptual framework. Perceived understandability, perceived technical competence, and cognition-based trust were investigated in this study.
Figure 2
Figure 2
Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) clinical decision support system.
Figure 3
Figure 3
Themes, categories, and subcategories of factors influencing trust. CDSS: clinical decision support system.
See this image and copyright information in PMC

References

    1. Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb Med Inform. 2016 Aug 02;Suppl 1:S103–16. doi: 10.15265/IYS-2016-s034. http://www.thieme-connect.com/DOI/DOI?10.15265/IYS-2016-s034 me2016-s034 - DOI - PMC - PubMed
    1. Tonekaboni S, Joshi S, McCradden M, Goldenberg A. What clinicians want: contextualizing explainable machine learning for clinical end use. Proceedings of Machine Learning Research; Machine Learning for Healthcare; Aug 8-10, 2019; Ann Arbor, MI. 2019. http://proceedings.mlr.press/v106/tonekaboni19a/tonekaboni19a.pdf
    1. Wang D, Yang Q, Abdul A, Lim B. Designing theory-driven user-centric explainable AI. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems; CHI '19: CHI Conference on Human Factors in Computing Systems; May 4 - 9, 2019; Glasgow Scotland UK. 2019. - DOI
    1. Holzinger A, Biemann C, Pattichis C, Kell D. What do we need to build explainable AI systems for the medical domain? arXiv. 2017:1–28. http://arxiv.org/abs/1712.09923
    1. The European Parliament and the Council of the European Union Chapter 3 Rights of the data subject. Official Journal of the European Union. 2016 Apr 27;:39–47. doi: 10.1093/oso/9780198826491.003.0147. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679 - DOI

LinkOut - more resources