Mitigating Interobserver Variability in Radiomics with ComBat: A Feasibility Study

Abstract

This study investigates Intraobserver Features Variability (IFV) in radiomics studies and assesses the effectiveness of the ComBat harmonization method in mitigating these effects. Methods: This study utilizes data from the NSCLC-Radiomics-Interobserver1 dataset, comprising CT scans of 22 Non-Small Cell Lung Cancer (NSCLC) patients, with multiple Gross Tumor Volume (GTV) delineations performed by five radiation oncologists. Segmentation was completed manually ("vis") or by autosegmentation with manual editing ("auto"). A total of 1229 radiomic features were extracted from each GTV, segmentation method, and oncologist. Features extracted included first order, shape, GLCM, GLRLM, GLSZM, and GLDM from original, wavelet-filtered, and LoG-filtered images. Results: Before implementing ComBat harmonization, 83% of features exhibited p-values below 0.05 in the "vis" approach; this percentage decreased to 34% post-harmonization. Similarly, for the "auto" approach, 75% of features demonstrated statistical significance prior to ComBat, but this figure declined to 33% after its application. Among a subset of three expert radiation oncologists, percentages changed from 77% to 25% for "vis" contouring and from 64% to 23% for "auto" contouring. This study demonstrates that ComBat harmonization could effectively reduce IFV, enhancing the feasibility of multicenter radiomics studies. It also highlights the significant impact of physician experience on radiomics analysis outcomes.

Keywords: batch correction; clinical imaging; multicenter studies; precision medicine; radiomics; segmentation.

Figure 1

Example of “vis” and “auto” GTV-1 contouring (patient 8) for each radiation oncologist. NSCLC-Radiomics-Interobserver1 dataset [32,33,34,35].

Figure 2

Radiomics workflow. CT images contoured by five different radiation oncologists, employing two distinct approaches: “vis” and “auto”. Feature extraction encompassed both original, wavelet-filtered, and LoG-filtered images, incorporating shape, first order, Gray-Level Co-occurrence Matrix (GLCM), Gray-Level Run-Length Matrix (GLRLM), Gray-Level Size Zone Matrix (GLSZM), and Gray-Level Dependence Matrix (GLDM) features. A post-extraction ComBat correction was conducted to reduce Interobserver Features Variability (IFV).

Figure 3

Density plots illustrating the distribution of “log-sigma-5-0-mm-3D_glszm_HighGrayLevelZoneEmphasis” before (left) and after (right) applying ComBat for the “vis” (upper) and “auto” (lower) contouring approaches. The data are segregated for five radiation oncologists: radiation oncologist 1 (in red), 2 (in blue), 3 (in green), 4 (in purple), and 5 (in orange). The data are aligned on a virtual site for comparison, showcasing changes in the feature’s distribution due to the ComBat transformation.

Figure 4

Whole sample (radiation oncologists 1–5) p-value boxplots. Comparison of radiomic feature percentages, categorized by image type (original, wavelet, LoG) and feature classes (Shape, first order, GLCM, GLRLM, GLSZM, GLDM), alongside p-values obtained from the Friedman test (Benjamini–Hochberg corrected). The dashed red line represents the significance threshold (p-value = 0.05). Panels (a,b) display the data before ComBat normalization, while panels (c,d) showcase the data after normalization. The “vis” segmentation approach is depicted in a red color scale, while the “auto” segmentation approach is represented with a blue color scale.

Figure 5

Expert radiation oncologist (radiation oncologist 2, 4, and 5) p-value boxplots. Comparison of radiomic feature percentages, categorized by image type (original, wavelet, LoG) and feature classes (Shape, first order, GLCM, GLRLM, GLSZM, GLDM), alongside p-values obtained from the Friedman test (Benjamini–Hochberg corrected). The dashed red line represents the significance threshold (p-value = 0.05). Panels (a,b) display the data before ComBat normalization, while panels (c,d) showcase the data after normalization. The “vis” segmentation approach is depicted in a purple color scale, while the “auto” segmentation approach is represented with a grey color scale.