Uncertainties in outcome modelling in radiation oncology

Abstract

Outcome models predicting e.g. survival, tumour control or radiation-induced toxicities play an important role in the field of radiation oncology. These models aim to support the clinical decision making and pave the way towards personalised treatment. Both validity and reliability of their output are required to facilitate clinical integration. However, models are influenced by uncertainties, arising from data used for model development and model parameters, among others. Therefore, quantifying model uncertainties and addressing their causes promotes the creation of models that are sufficiently reliable for clinical use. This topical review aims to summarise different types and possible sources of uncertainties, presents uncertainty quantification methods applicable to various modelling approaches, and highlights central challenges that need to be addressed in the future.

Keywords: Outcome; Radiation oncology; Statistical analyses; Uncertainty modelling.

Graphical abstract

Fig. 1

Concept of data domains for addressing different types of uncertainties. (A) During the training process a mapping f(x) between the data points in the data domain x_i ∈ X and the outcome domain y_i ∈ Y is created, e.g. using logistic regression or a neural network. (B) When applying a model to unseen data, in-domain uncertainty refers to uncertainties for data points which are in the observed domain and have characteristics similar to the training data. In contrast, applying the model to data whose characteristics differ from the samples of the observed domain can lead to out-of-domain uncertainty. (C) In order to achieve a model that can generalise well, capturing and reducing the true domain are two main strategies for reducing uncertainties.

Fig. 2

Aleatoric and epistemic uncertainty. Models 1 and 2 are trained using the same data points, but their predictions are influenced by aleatoric and epistemic uncertainties. For a classification task (A), the aleatoric uncertainty is high in the domain with overlapping data points due to inherent randomness of the data, whereas the epistemic uncertainty is high in areas with missing data points. Similarly, for a regression task (B), the epistemic uncertainty is high in areas with no or only sparse data points. In areas with numerous data points, inherent randomness results in a high aleatoric uncertainty.

Fig. 3

Important elements of the data collection process. Main study designs (top, green), central questions related to data collection requirements (middle, blue) as well as relevant strategies (bottom, orange) that can be used to create a reliable dataset are shown. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

Data analysis plan. A data analysis plan contains several steps that need to be addressed during the creation of a dataset for outcome modelling. The individual steps build on each other: First, in- and exclusion parameters are needed, defining the characteristics of the cohort. Second, protocols, parameters and data storage for data acquisition must be defined. Third, methods for data curation including analysis, sorting and validation of the content must be outlined. Fourth, data annotation and processing steps need to be defined, e.g. for segmentation or feature selection.

Fig. 5

Methods for uncertainty quantification. Illustration of the basic principles for four different methods to quantify uncertainty using the example of a simple neural network. The methods can be used to model the relationship y = f(x) and to calculate a measure of uncertainty. Here, the variance σ² is assessed. (A) Ensemble methods use several different networks. The prediction and a corresponding measure of uncertainty are calculated from the individual predictions. (B) Bayesian methods are based on Bayesian inference, resulting in a non-discrete model output. The resulting distribution of the outcome variable is used to calculate for instance the mean and variance. (C) Deterministic methods predict the outcome and their corresponding uncertainty simultaneously in a single-forward pass, e.g. using a connected second network in case of an external method. (D) In test-time augmentation methods, the input variable is modified resulting in several output variables. Graphical illustration after Gawlikowski et al. [18].

Fig. 6

Use and representation of uncertainty quantification. Examples for applications and uncertainty representation methods are shown in the top row. These methods can be used to increase model interpretability, to address subpar model performance or to raise the need for expert referral, as illustrated in the bottom row. Abbreviations: NTCP: Normal tissue complication probability, crit.: critical value, Exp. conf.: Expected confidence, Obs. conf.: Observed confidence, fx.: fraction, Non-conf.: Non-conformity, y_90%: 90% confidence prediction set.