Causality matters in medical imaging

Abstract

Causal reasoning can shed new light on the major challenges in machine learning for medical imaging: scarcity of high-quality annotated data and mismatch between the development dataset and the target environment. A causal perspective on these issues allows decisions about data collection, annotation, preprocessing, and learning strategies to be made and scrutinized more transparently, while providing a detailed categorisation of potential biases and mitigation techniques. Along with worked clinical examples, we highlight the importance of establishing the causal relationship between images and their annotations, and offer step-by-step recommendations for future studies.

Fig. 1. Key challenges in machine learning for medical imaging.

a Data scarcity and b–d data mismatch. X represents images and Y, annotations (e.g. diagnosis labels). P_tr refers to the distribution of data available for training a predictive model, and P_te is the test distribution, i.e. data that will be encountered once the model is deployed. Dots represent data points with any label, while circles and crosses indicate images with different labels (e.g. cases vs. controls).

Fig. 2. Causal diagrams for medical imaging examples.

a Skin lesion classification. b Prostate tumour segmentation. Filled circular nodes represent measured variables, double circular nodes denote sample selection indicators, and squares are used for sample domain indicators. Here we additionally highlight the direction of the predictive task.

Fig. 3. Selection diagrams for dataset shift.

a–c Causal and d–f anticausal scenarios, with corresponding factorisations of the joint distribution P_D(X, Y, Z). X is the acquired image; Y, the prediction target; Z, the unobserved true anatomy; and D, the domain indicator (0: ‘train’, 1: ‘test’). An unfilled node means the variable is unmeasured.

Fig. 4. Causal diagrams for different sample selection scenarios.

a Random; b image-dependent; c target-dependent; d jointly dependent. S = 1 indicates an observed sample, and plain edges represent either direction.

Fig. 5. A ‘scaffold’ causal diagram summarising typical medical imaging workflows.

We believe most practical cases can be adapted from this generic structure by removing or adding elements. Here are represented a variety of possible prediction targets (marked Y₁–Y₄): some anticausal (Y₁, Y₂) and others, causal (Y₃, Y₄). ‘Annotation’ here refers to any image-derived data, such as lesion descriptions, regions of interest, spatial landmark coordinates, or segmentation maps. Note that annotators will often be aware of the patients' records and diagnoses, in which cases there could be additional arrows from Y₁ or Y₂ towards Y₄.