Quantitative imaging biomarkers: a review of statistical methods for technical performance assessment

Abstract

Technological developments and greater rigor in the quantitative measurement of biological features in medical images have given rise to an increased interest in using quantitative imaging biomarkers to measure changes in these features. Critical to the performance of a quantitative imaging biomarker in preclinical or clinical settings are three primary metrology areas of interest: measurement linearity and bias, repeatability, and the ability to consistently reproduce equivalent results when conditions change, as would be expected in any clinical trial. Unfortunately, performance studies to date differ greatly in designs, analysis method, and metrics used to assess a quantitative imaging biomarker for clinical use. It is therefore difficult or not possible to integrate results from different studies or to use reported results to design studies. The Radiological Society of North America and the Quantitative Imaging Biomarker Alliance with technical, radiological, and statistical experts developed a set of technical performance analysis methods, metrics, and study designs that provide terminology, metrics, and methods consistent with widely accepted metrological standards. This document provides a consistent framework for the conduct and evaluation of quantitative imaging biomarker performance studies so that results from multiple studies can be compared, contrasted, or combined.

Keywords: agreement; bias; imaging biomarkers; linearity; precision; quantitative imaging; reliability; repeatability; reproducibility.

Figure 1

The Quantitative Image Biomarker analytical process.

Figure 2

Plot of Phantom Volumes versus Measured volumes with identity line (dashed black). Values were taken from QIBA 3A data on phantom volumes and augmented with simulated data for illustration purposes.

Figure 3

Measuring interval analysis of bias. Note that the values measured beyond the measuring interval are necessary to define the interval.

Figure 4

Bias plotted against Truth for the relationship Y=1.4*X+ε with σ²=1.0

Figure 5

QIB measurement variability plots for comparison to the reference values

Figure 6

Linearity Example derived from the QIBA 3A challenge

Figure 7

Bland-Altman-like plot example of agreement when a reference is available

Figure 8

Bland-Altman-like plot for analysis of repeatability using QIBA publicly available data when reference values are not available (see 11.1.1 for data). All original volumes are in mm³.

Figure 9

Relationship of sample size to rSEVw for k= 2,3,4. rSEVw values as a function of the total number of N measurements are also shown for N= 32, 48, 60, 120.

Figure 10

Phantom CT volumetry comparison of two methods plotted for logtransformed data (original units- mm³). Figure (a) slope=0.974, intercept= 0.086. Figure (b) shows the Bland-Altman plot with Lower and Upper agreement limits. Correlation = 0.99; RDC=0.234

Figure 11

Box-Whisker Plot for multigroup reproducibility. Points are jittered for viewing. Whiskers indicate 1.5*IQR outlier boundary