Interobserver Variability in Manual Versus Semi-Automatic CT Assessments of Small Lung Nodule Diameter and Volume

Abstract

Background: This study aimed to assess the interobserver variability of semi-automatic diameter and volumetric measurements versus manual diameter measurements for small lung nodules identified on computed tomography scans.

Methods: The radiological patient database was searched for CT thorax examinations with at least one noncalcified solid nodule (∼3-10 mm). Three radiologists with four to six years of experience evaluated each nodule in accordance with the Fleischner Society guidelines using standard diameter measurements, semi-automatic lesion diameter measurements, and volumetric assessments. Spearman's correlation coefficient measured intermeasurement agreement. We used descriptive Bland-Altman plots to visualize agreement in the measured data. Potential discrepancies were analyzed.

Results: We studied a total of twenty-six nodules. Spearman's test showed that there was a much stronger relationship (p < 0.05) between reviewers for the semi-automatic diameter and volume measurements (avg. r = 0.97 ± 0.017 and 0.99 ± 0.005, respectively) than for the manual method (avg. r = 0.91 ± 0.017). In the Bland-Altman test, the semi-automatic diameter measure outperformed the manual method for all comparisons, while the volumetric method had better results in two out of three comparisons. The incidence of reviewers modifying the software's automatic outline varied between 62% and 92%.

Conclusions: Semi-automatic techniques significantly reduced interobserver variability for small solid nodules, which has important implications for diagnostic assessments and screening. Both the semi-automatic diameter and semi-automatic volume measurements showed improvements over the manual measurement approach. Training could further diminish observer variability, given the considerable diversity in the number of adjustments among reviewers.

Keywords: computed tomography (CT); interobserver variability; lung nodule size; semi-automatic measurement.

Figure 1

Illustration according to the Fleishner Society of the primary lesion categories, with an emphasis on solid lung nodules measuring roughly 3–10 mm, which are the subject of the current study.

Figure 2

Flowchart illustrating the procedure for semi-automated review.

Figure 3

Spearman and Bland–Altman plots demonstrating agreement between reviewers 1 and 2 in manual (a,d) and semi-automatic (b,e) mean diameter and volumetric measurements (c,f).

Figure 4

Spearman and Bland–Altman plots demonstrating agreement between reviewers 1 and 3 in manual (a,d) and semi-automatic (b,e) mean diameter and volumetric measurements (c,f).

Figure 5

Spearman and Bland–Altman plots demonstrating agreement between reviewers 2 and 3 in manual (a,d) and semi-automatic (b,e) mean diameter and volumetric measurements (c,f).

Figure 5

Spearman and Bland–Altman plots demonstrating agreement between reviewers 2 and 3 in manual (a,d) and semi-automatic (b,e) mean diameter and volumetric measurements (c,f).

Figure 6

One of the smaller nodules in the study, which is shown by the white arrow, exhibited a notable size discrepancy as assessed by reviewer 3. The manual measurement (a) was 2.8 × 2.8 mm (white lines), while the semi-automatic measurement (b) yielded greater values of 3.7 × 3.2 mm (the blue line denotes the longest diameter; while the red line denotes the smallest diameter).

Figure 7

A prominently discernible spiculated nodule. Reviewer 1 performed manual measurements in axial (a1), coronal (a2), and sagittal (a3) sections. The largest diameter was measured in the axial section (a1) and was found to be 9.4 × 5.2 mm. Reviewer 1 utilized the semi-automatic software (Vitrea™) (b1–b3) to measure the same nodule and obtained a volume of 236.4 mm³ (diameter: 10.3 × 7.7 mm) (b1–b3). Subsequently, following modifications in the nodule delineation (c1–c3), the volume measured 228.6 mm³ (diameter: 9.5 × 6.8 mm).

Figure 8

A complicated situation where the nodule partially blended together with blood vessels in certain CT slices. This caused the delineation to be elongated and partially incorrect, where both the nodule and parts of the blood vessel (on the right) were included in the initial estimate made by the semi-automatic algorithm ((a); volume: 431 mm³, dimensions: 11.2 × 7.5 mm; blue-colored lines represent the longest diameter). The reviewers adjusted the delineation to accurately depict the size of the nodule ((b); reviewer 3 after adjustments; volume: 142.8 mm³, diameter: 7.6 × 7.3 mm).