To warrant clinical adoption AI models require a multi-faceted implementation evaluation

Abstract

Despite artificial intelligence (AI) technology progresses at unprecedented rate, our ability to translate these advancements into clinical value and adoption at the bedside remains comparatively limited. This paper reviews the current use of implementation outcomes in randomized controlled trials evaluating AI-based clinical decision support and found limited adoption. To advance trust and clinical adoption of AI, there is a need to bridge the gap between traditional quantitative metrics and implementation outcomes to better grasp the reasons behind the success or failure of AI systems and improve their translation into clinical value.

Fig. 1. Flow diagram of the study review process and the exclusion of studies.

Randomized controlled trials evaluating the effectiveness of artificial intelligence-based clinical decision support systems in real-world clinical settings were extracted from previous systematic reviews by Zhou et al. and Plana et al..

Fig. 2. Overview of the current and desired approach to evaluate artificial intelligence in healthcare.

a In the current situation, artificial intelligence-based clinical decision support systems (AI-CDSS), are clinically deployed, after going through multiple preclinical validations (e.g., external and temporal algorithm validation) to assess their clinical utility and effectiveness. b To enhance comprehension of factors that contributed to successful implementation or failure at the bedside, implementation outcomes should be systematically integrated in future clinical trials evaluating AICDSS in real-world clinical settings. *Implementation outcomes as described by Proctor et al..