Recognition performance and perceived quality of video enhanced for the visually impaired

Abstract

Subjects with central field loss (CFL) individually selected enhancement parameters to improve visibility of static video images. The effect of enhancement on performance and on perceived quality of motion video was assessed. Performance, e.g. recognition of visual details, was assessed by having subjects answer questions regarding visual information contained in the video motion segments that were enhanced using the individually-selected parameters. Enhancement did not improve subject performance on questions about video content. This result might be due to a ceiling effect limitation of the performance assessment method. In a second procedure, subjects' continuous perceptions of quality (using an adjective-based rating scale) were made while the enhancement parameters were abruptly switched among multiple values; these included the individually-selected enhancements, as well as unenhanced, over-enhanced, and degraded segments. The results indicate that adaptive enhancement (individually-tuned using a static image) adds significantly to perceived image quality when viewing motion video. Subjects who selected stronger contrast enhancement also perceived the enhancement to provide a larger benefit in image quality.

Figure 1

An example of (a) the original, unenhanced image and (b) an image whose detail and contrast settings have been enhanced in accordance with an individual subject's selection. In the experiment, images were presented in color on a 27″ television. Note the moderate level of enhancement selected and changes in local luminance (particularly on the trouser leg) that permit greater enhancement of high frequency details (such as the folds in the material).

Figure 2

Enhancement parameters chosen by each subject. The position of each cross corresponds to mean mouse position on the graphics tablet and the error bars represent SEM. The vertical line at contrast = 27 (arbitrary units) represents no enhancement (original image). Area to the left of that line represents image degradation by low-pass filtering. Area to the right represents enhancement. Area to the right of the shaded diamond represents over-enhancement. All subjects selected values corresponding to enhancement. (a) The combined group of 46 patients, 21 of whom averaged across 5 images, 2 criteria, and 2 repetitions, and 25 of whom averaged across 4 images and 2 repetitions. (b) The same data but only for the 20 subjects who completed the continuous evaluation of perceived quality part of the experiment. The filled symbols to the right of the vertical line represent enhancement settings used in the second part of the experiment for one subject. Diamond: individually selected enhancement settings; Triangles: settings used if in the B_× group; Squares: settings used if in the B₊ group; Open circles: settings that resulted in two degraded images; Filled circle: setting for the unenhanced original image resulted from any setting with contrast =27.

Figure 3

Frequency distribution of the continuous perceived image quality scores indicated by a subject using the graphic tablet in response to 4 of the test video segments: the original unprocessed video (setting 46 - 27 on the Digivision), the individually preferred enhancement (setting 46 - 62 for this patient), a degraded video (46 - 10) and an over-enhanced video (92-62). The subject responses were binned into 10 bins of 500 arbitrary graphic tablet units. This subject clearly preferred his individually selected enhancement and thus has one distribution (46-62) that is shifted to the left of the others. The over-enhanced distribution (92-62) is also shifted to the left, but not as much as the preferred enhancement. The distribution for the degraded set is highly shifted to the right. For simplicity, only four of the 8 distributions (corresponding to the 7 enhancement conditions and the original image tested for each subject) obtained are shown. Distributions from a subject such as shown in Figure 4b would not be clearly separated. The scores shown here were used to construct three corresponding ROC curves shown in Figure 4.

Figure 4

ROC data and fitted curves for differences in perceived image quality between the original and processed video segments. (a) Data for one subject (9+ from group B₊) who preferred all of the enhancement settings, but the perceived image quality was only significantly different from the original for the individually selected enhancement (circles) and one of the arbitrary enhancements (triangles). This subject rejected the degraded segments as compared to the original segments (but only the most degraded was significantly rejected). (b) This example shows a subject who did not perceive much difference in image quality between the processed video segments, including both the degraded and enhanced videos and the original video.

Figure 5

A_z (area under ROC curve) values for subject 9+, whose ROC curves are shown in Figure 4a. A_z values were significantly higher than 0.5 for the individually selected enhancement and one of the arbitrary enhancements indicating that perceived image quality for these two enhancements was significantly different from the original. Error bars are 95% confidence levels as determined from the Rockit program.

Figure 6

Average A_z for all subjects. The three filled points show the two degraded conditions and the individually-selected enhancement condition, which were common across all 20 subjects. The degraded conditions have average A_z less than 0.5 whereas the enhanced condition has an average A_z greater than 0.5. The other points show all the other enhanced conditions for the (B₊) and the (B_×) groups of 10 patients each. The (×) and (+) symbols refer to the set of conditions presented as described in the text. For all except two of these enhancement conditions, the A_z is significantly greater than 0.5 as indicated by lower bound of the error bars (SEM).