A new perceptual difference model for diagnostically relevant quantitative image quality evaluation: a preliminary study

Abstract

Purpose: Most objective image quality metrics average over a wide range of image degradations. However, human clinicians demonstrate bias toward different types of artifacts. Here, we aim to create a perceptual difference model based on Case-PDM that mimics the preference of human observers toward different artifacts.

Method: We measured artifact disturbance to observers and calibrated the novel perceptual difference model (PDM). To tune the new model, which we call Artifact-PDM, degradations were synthetically added to three healthy brain MR data sets. Four types of artifacts (noise, blur, aliasing or "oil painting" which shows up as flattened, over-smoothened regions) of standard compressed sensing (CS) reconstruction, within a reasonable range of artifact severity, as measured by both PDM and visual inspection, were considered. After the model parameters were tuned by each synthetic image, we used a functional measurement theory pair-comparison experiment to measure the disturbance of each artifact to human observers and determine the weights of each artifact's PDM score. To validate Artifact-PDM, human ratings obtained from a Double Stimulus Continuous Quality Scale experiment were compared to the model for noise, blur, aliasing, oil painting and overall qualities using a large set of CS-reconstructed MR images of varying quality. Finally, we used this new approach to compare CS to GRAPPA, a parallel MRI reconstruction algorithm.

Results: We found that, for the same Artifact-PDM score, the human observer found incoherent aliasing to be the most disturbing and noise the least. Artifact-PDM results were highly correlated to human observers in both experiments. Optimized CS reconstruction quality compared favorably to GRAPPA's for the same sampling ratio.

Conclusions: We conclude our novel metric can faithfully represent human observer artifact evaluation and can be useful in evaluating CS and GRAPPA reconstruction algorithms, especially in studying artifact trade-offs.

Fig. 1

Raw images used for human observer experiments. (a) is Transverse, (b) Coronal, and (c) Sagittal. Their MR raw data were used for both CS and PPI reconstruction.

Fig. 2

Artifact disturbance calibration lines obtained from FMT experiment for the Transverse (a), Coronal (b) and Sagittal (c) data sets. For the same artifact severity of noise, blur, aliasing, or oil-painting degradation pattern, as measured by Artifact-PDM, their disturbances to human observer distribute differently, with noise being the least disturbing. One could observe that the Artifact-PDM shows good correlation to artifact scores, but the relationship is not linear except for the noise.

Fig. 3

Individual ratings from one radiologist and four engineers against Aliasing-PDM predictions when they evaluated Transverse brain data set during the FMT experiment.

Fig. 4

Linear correlation between Artifact-PDM and observers when evaluating aliasing artifact. Data are from CS reconstructions of Sagittal image in the DSCQS experiment. Cross points represent responses from subject Eng-1, “x” points represent responses from subject Eng-2, and open circle points represent average responses. The average human subject data (open circle) were fitted to y = ax + b, and the functions were represented by the straight lines in the figure.

Fig. 5

Comparison of MR reconstruction algorithms for the Transverse data set. (a) shows different artifact scores for CS, ZP and PPI reconstructions at different sampling rate. (b) shows comprehensive score for the three methods. CS was ranked the best by both selective artifact evaluation and overall quality evaluation.

Fig. 6

Algorithm comparison for MR reconstructions using only 50% data. (a) is a reference image. (b)–(d) correspond to images reconstructed by ZF, CS and GRAPPA, respectively.