Prediction of human observer performance in a 2-alternative forced choice low-contrast detection task using channelized Hotelling observer: impact of radiation dose and reconstruction algorithms

Abstract

Purpose: Efficient optimization of CT protocols demands a quantitative approach to predicting human observer performance on specific tasks at various scan and reconstruction settings. The goal of this work was to investigate how well a channelized Hotelling observer (CHO) can predict human observer performance on 2-alternative forced choice (2AFC) lesion-detection tasks at various dose levels and two different reconstruction algorithms: a filtered-backprojection (FBP) and an iterative reconstruction (IR) method.

Methods: A 35 × 26 cm(2) torso-shaped phantom filled with water was used to simulate an average-sized patient. Three rods with different diameters (small: 3 mm; medium: 5 mm; large: 9 mm) were placed in the center region of the phantom to simulate small, medium, and large lesions. The contrast relative to background was -15 HU at 120 kV. The phantom was scanned 100 times using automatic exposure control each at 60, 120, 240, 360, and 480 quality reference mAs on a 128-slice scanner. After removing the three rods, the water phantom was again scanned 100 times to provide signal-absent background images at the exact same locations. By extracting regions of interest around the three rods and on the signal-absent images, the authors generated 21 2AFC studies. Each 2AFC study had 100 trials, with each trial consisting of a signal-present image and a signal-absent image side-by-side in randomized order. In total, 2100 trials were presented to both the model and human observers. Four medical physicists acted as human observers. For the model observer, the authors used a CHO with Gabor channels, which involves six channel passbands, five orientations, and two phases, leading to a total of 60 channels. The performance predicted by the CHO was compared with that obtained by four medical physicists at each 2AFC study.

Results: The human and model observers were highly correlated at each dose level for each lesion size for both FBP and IR. The Pearson's product-moment correlation coefficients were 0.986 [95% confidence interval (CI): 0.958-0.996] for FBP and 0.985 (95% CI: 0.863-0.998) for IR. Bland-Altman plots showed excellent agreement for all dose levels and lesions sizes with a mean absolute difference of 1.0% ± 1.1% for FBP and 2.1% ± 3.3% for IR.

Conclusions: Human observer performance on a 2AFC lesion detection task in CT with a uniform background can be accurately predicted by a CHO model observer at different radiation dose levels and for both FBP and IR methods.

Figure 1

Phantom setup. A 35 × 26 cm² torso-shaped phantom filled with water was used to simulate the abdomen of an average-sized patient. Three rods with different diameters (small: 3 mm; medium: 5 mm; large: 9 mm) were placed in the center region of the water tank (arrows). The acrylic resolution target was used only to hold the rods in position and was not included in the evaluated images.

Figure 2

A collage of images with no, small (3 mm), medium (5 mm), or large (9 mm) lesions at different mAs settings. The display window level and width are 40 and 300 HU, respectively.

Figure 3

Twenty-one 2AFC studies (FBP: five mAs settings × three lesion sizes; IR: two mAs settings × three lesion sizes) were generated by extracting a small region of interest around the lesion and at the corresponding location on the background image. Each 2AFC study had 100 trials obtained from repeated scans, totaling 2100 trials.

Figure 4

Garbor filters with six channel passbands, five orientations, and two phases. (a) 30 channels when phase equals zero. (b) 30 channels when phase equals π/2.

Figure 5

A flowchart on how the CHO makes a decision for each 2AFC trial.

Figure 6

For medium size (5 mm) lesion and 120 mAs, a calibration of internal noise was performed. The final internal noise was determined to be 9.35 times the noise of the decision variable when signal was absent.

Figure 7

Percent correct in each of the 15 2AFC tasks obtained by human observers (filled square symbols) and predicted by the CHO model observer (empty square symbols). The 15 2AFC tasks were generated at five mAs levels (60, 120, 240, 360, and 480 mAs) and three lesion sizes (small, medium, and large).

Figure 8

Bland-Altman plot of percent correct difference between human and model observers in the 15 2AFC tasks for FBP reconstruction. The two solid lines (−3.3% and 2.4%) indicate the average difference ±2σ, where σ is the standard deviation of the differences.

Figure 9

Performance comparison between human observers (filled square symbols) and model observers (empty square symbols) for the six 2AFC tasks when IR reconstruction was applied. The six 2AFC tasks were generated at two mAs levels (60 and 120 mAs) and three lesion sizes (small, medium, and large). The performance for the 2AFC tasks when FBP reconstruction was used was also displayed as a reference.

Figure 10

Bland-Altman plot of percent correct difference between human and model observers in all 21 2AFC tasks. The two solid lines (−3.2% and 2.2%) indicate the average difference ±2σ, where σ is the standard deviation of the differences. For the six points with IR, the average difference ±2 σ is [−8.8%, 5.2%]. If excluding the only point with a big difference of −8.6% (120 mAs and small lesion), the average difference ±2 σ is [−3.0%, 2.1%], similar to FBP.