Quantitative and Qualitative Evaluation of Provider Use of a Novel Machine Learning Model for Favorable Outcome Prediction

Abstract

Predictive models may be particularly beneficial to clinicians when they face uncertainty and seek to develop a mental model of disease progression, but we know little about the post-implementation effects of predictive models on clinicians' experience of their work. Combining survey and interview methods, we found that providers using a predictive algorithm reported being significantly less uncertain and better able to anticipate, plan and prepare for patient discharge than non-users. The tool helped hospitalists form and develop confidence in their mental models of a novel disease (Covid-19). Yet providers' attention to the predictive tool declined as their confidence in their own mental models grew. Predictive algorithms that not only offer data but also provide feedback on decisions, thus supporting providers' motivation for continuous learning, hold promise for more sustained provider attention and cognition augmentation.