Making machine learning matter to clinicians: model actionability in medical decision-making

Abstract

Machine learning (ML) has the potential to transform patient care and outcomes. However, there are important differences between measuring the performance of ML models in silico and usefulness at the point of care. One lens to use to evaluate models during early development is actionability, which is currently undervalued. We propose a metric for actionability intended to be used before the evaluation of calibration and ultimately decision curve analysis and calculation of net benefit. Our metric should be viewed as part of an overarching effort to increase the number of pragmatic tools that identify a model's possible clinical impacts.

Fig. 1. A conceptual schematic illustrating the typical relationship between machine learning actionability and entropy.

Actionability typically increases with decreasing entropy of the diagnostic possibility probability distribution and/or conditional future state probability distribution during key phases of medical decision-making. S1 State 1, S2 State 2, S3 State 3, S4 State 4, Sn the Nth State.