Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Feb 3:2023:9919269.
doi: 10.1155/2023/9919269. eCollection 2023.

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Qian Xu  1   2   3   4   5 , Wenzhao Xie  4 , Bolin Liao  3 , Chao Hu  6 , Lu Qin  2 , Zhengzijin Yang  2 , Huan Xiong  2 , Yi Lyu  2 , Yue Zhou  2 , Aijing Luo  1   4   5
Affiliations

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Qian Xu et al. J Healthc Eng. .

Abstract

Background: Artificial intelligence (AI) has developed rapidly, and its application extends to clinical decision support system (CDSS) for improving healthcare quality. However, the interpretability of AI-driven CDSS poses significant challenges to widespread application.

Objective: This study is a review of the knowledge-based and data-based CDSS literature regarding interpretability in health care. It highlights the relevance of interpretability for CDSS and the area for improvement from technological and medical perspectives.

Methods: A systematic search was conducted on the interpretability-related literature published from 2011 to 2020 and indexed in the five databases: Web of Science, PubMed, ScienceDirect, Cochrane, and Scopus. Journal articles that focus on the interpretability of CDSS were included for analysis. Experienced researchers also participated in manually reviewing the selected articles for inclusion/exclusion and categorization.

Results: Based on the inclusion and exclusion criteria, 20 articles from 16 journals were finally selected for this review. Interpretability, which means a transparent structure of the model, a clear relationship between input and output, and explainability of artificial intelligence algorithms, is essential for CDSS application in the healthcare setting. Methods for improving the interpretability of CDSS include ante-hoc methods such as fuzzy logic, decision rules, logistic regression, decision trees for knowledge-based AI, and white box models, post hoc methods such as feature importance, sensitivity analysis, visualization, and activation maximization for black box models. A number of factors, such as data type, biomarkers, human-AI interaction, needs of clinicians, and patients, can affect the interpretability of CDSS.

Conclusions: The review explores the meaning of the interpretability of CDSS and summarizes the current methods for improving interpretability from technological and medical perspectives. The results contribute to the understanding of the interpretability of CDSS based on AI in health care. Future studies should focus on establishing formalism for defining interpretability, identifying the properties of interpretability, and developing an appropriate and objective metric for interpretability; in addition, the user's demand for interpretability and how to express and provide explanations are also the directions for future research.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no conflicts of interest.

Figures

Figure 1
Figure 1
Diagram of literature search and selection processes.
See this image and copyright information in PMC

References

    1. Brigl B. Section 5: decision support, knowledge representation and management: decision support, knowledge representation and management: a broad methodological spectrum. Yearbook of Medical Informatics . 2006;15(1):81–83. doi: 10.1055/s-0038-1638472. - DOI - PubMed
    1. Müller L., Gangadharaiah R., Klein S. C., et al. An open access medical knowledge base for community driven diagnostic decision support system development. BMC Medical Informatics and Decision Making . 2019;19(1):p. 93. doi: 10.1186/s12911-019-0804-1. - DOI - PMC - PubMed
    1. Gaw N., Schwedt T. J., Chong C. D., Wu T., Li J. A clinical decision support system using multi-modality imaging data for disease diagnosis. IISE Transactions on Healthcare Systems Engineering . 2018;8(1):36–46. doi: 10.1080/24725579.2017.1403520. - DOI
    1. Liu Y., Wang Z., Ren J., et al. A covid-19 risk assessment decision support system for general practitioners: design and development study. Journal of Medical Internet Research . 2020;22(6) doi: 10.2196/19786.e19786 - DOI - PMC - PubMed
    1. Griffey R. T., Lo H. G., Burdick E., Keohane C., Bates D. W. 265: guided medication dosing for elderly emergency department patients using a real-time, computerized decision support tool. Annals of Emergency Medicine . 2009;54(3):p. S83. doi: 10.1016/j.annemergmed.2009.06.295. - DOI

Publication types

LinkOut - more resources