Conformal Prediction in Clinical Medical Sciences

Abstract

The use of machine learning (ML) and artificial intelligence (AI) applications in medicine has attracted a great deal of attention in the medical literature, but little is known about how to use Conformal Predictions (CP) to assess the accuracy of individual predictions in clinical applications. We performed a comprehensive search in SCOPUS® to find papers reporting the use of CP in clinical applications. We identified 14 papers reporting the use of CP for clinical applications, and we briefly describe the methods and results reported in these papers. The literature reviewed shows that CP methods can be used in clinical applications to provide important insight into the accuracy of individual predictions. Unfortunately, the review also shows that most of the studies have been performed in isolation, without input from practicing clinicians, not providing comparisons among different approaches and not considering important socio-technical considerations leading to clinical adoption.

Keywords: Artificial intelligence in medicine; Conformal Prediction, Predictive analytics; Uncertainty quantification.

Fig. 1.

Archetypical pseudocode for CP implementation

Fig. 2.

SCOPUS Query used in this review

Fig. 3.

Example of the matrix that can be constructed to inform the probability of non-conformal predictions for each pair of conditions considered. From ref. [21]

Fig. 4.

Visual representation of the Conformal Prediction results of the classification of mass spectroscopy traces used for cancer informatics in reference [30]