Informatics in radiology: what can you see in a single glance and how might this guide visual search in medical images?

Abstract

Diagnostic accuracy for radiologists is above that expected by chance when they are exposed to a chest radiograph for only one-fifth of a second, a period too brief for more than a single voluntary eye movement. How do radiologists glean information from a first glance at an image? It is thought that this expert impression of the gestalt of an image is related to the everyday, immediate visual understanding of the gist of a scene. Several high-speed mechanisms guide our search of complex images. Guidance by basic features (such as color) requires no learning, whereas guidance by complex scene properties is learned. It is probable that both hardwired guidance by basic features and learned guidance by scene structure become part of radiologists' expertise. Search in scenes may be best explained by a two-pathway model: Object recognition is performed via a selective pathway in which candidate targets must be individually selected for recognition. A second, nonselective pathway extracts information from global or statistical information without selecting specific objects. An appreciation of the role of nonselective processing may be particularly useful for understanding what separates novice from expert radiologists and could help establish new methods of physician training based on medical image perception.

Figure 1

Two-pathway architecture for visual processing. Diagram and medical image show the nonselective (red arrow, red circle) and selective (blue line, blue arrows) pathways of visual search. The selective pathway can combine features and recognize objects. However, it is capacity limited; it is difficult for it to process more than one item or location at a time. We posit that searches in everyday scenes and medical images are guided by the nonselective pathway, which can extract statistics from the entire scene or image. This global information can be used to categorize scenes (eg, as natural or man-made) and their contents (eg, an animal or a tumor). Nonselective processing does not support precise object recognition.

Figure 2a

Visual search of random arrays versus real scenes. (a, b) Search for a framed picture. (a) In classic visual search experiments, random arrays of items are presented, deliberately eliminating the possibility of guidance of search by scene context or scene structure. (b) In real scenes, context and structure drastically constrain where we search. In this case, search will start on the walls, where pictures most commonly hang. (c, d) Search for a lung nodule. (c) If the background and spatial layout of a medical image are removed, search becomes much more difficult. (d) With a real medical image, nonselective scene processing will help expert radiologists find lung nodules quickly and with fewer extraneous eye movements.

Figure 2b

Visual search of random arrays versus real scenes. (a, b) Search for a framed picture. (a) In classic visual search experiments, random arrays of items are presented, deliberately eliminating the possibility of guidance of search by scene context or scene structure. (b) In real scenes, context and structure drastically constrain where we search. In this case, search will start on the walls, where pictures most commonly hang. (c, d) Search for a lung nodule. (c) If the background and spatial layout of a medical image are removed, search becomes much more difficult. (d) With a real medical image, nonselective scene processing will help expert radiologists find lung nodules quickly and with fewer extraneous eye movements.

Figure 2c

Visual search of random arrays versus real scenes. (a, b) Search for a framed picture. (a) In classic visual search experiments, random arrays of items are presented, deliberately eliminating the possibility of guidance of search by scene context or scene structure. (b) In real scenes, context and structure drastically constrain where we search. In this case, search will start on the walls, where pictures most commonly hang. (c, d) Search for a lung nodule. (c) If the background and spatial layout of a medical image are removed, search becomes much more difficult. (d) With a real medical image, nonselective scene processing will help expert radiologists find lung nodules quickly and with fewer extraneous eye movements.

Figure 2d

Visual search of random arrays versus real scenes. (a, b) Search for a framed picture. (a) In classic visual search experiments, random arrays of items are presented, deliberately eliminating the possibility of guidance of search by scene context or scene structure. (b) In real scenes, context and structure drastically constrain where we search. In this case, search will start on the walls, where pictures most commonly hang. (c, d) Search for a lung nodule. (c) If the background and spatial layout of a medical image are removed, search becomes much more difficult. (d) With a real medical image, nonselective scene processing will help expert radiologists find lung nodules quickly and with fewer extraneous eye movements.

Figure 3

Experimental procedure in the study of Evans et al (36). Diagnostic performance was reliably above that expected by chance when radiologists classified mammograms after a very brief display. However, the ability to localize the lesion was not significantly above that expected by chance. The red arrows reflect the fact that the radiologists were able to move the selection bar to reflect confidence in their decision. pt = point.

Figure 4a

Scan paths in visual search. (a, b) Scan paths (red lines) for a first-year resident (a) and an expert radiologist (b) while searching a chest radiograph for lung nodules. (c, d) Analogous scan paths for naive observers searching artificial scene stimuli. Scene guidance allows the target to be quickly found in a realistic scene (d). However, when the background is removed and the object positions are scrambled, more eye movements and more time are necessary to find the target (c). (Figs 4a and 4b courtesy of Elizabeth Krupinski, PhD, University of Arizona College of Medicine, Tucson, Ariz.)

Figure 4b

Scan paths in visual search. (a, b) Scan paths (red lines) for a first-year resident (a) and an expert radiologist (b) while searching a chest radiograph for lung nodules. (c, d) Analogous scan paths for naive observers searching artificial scene stimuli. Scene guidance allows the target to be quickly found in a realistic scene (d). However, when the background is removed and the object positions are scrambled, more eye movements and more time are necessary to find the target (c). (Figs 4a and 4b courtesy of Elizabeth Krupinski, PhD, University of Arizona College of Medicine, Tucson, Ariz.)

Figure 4c

Scan paths in visual search. (a, b) Scan paths (red lines) for a first-year resident (a) and an expert radiologist (b) while searching a chest radiograph for lung nodules. (c, d) Analogous scan paths for naive observers searching artificial scene stimuli. Scene guidance allows the target to be quickly found in a realistic scene (d). However, when the background is removed and the object positions are scrambled, more eye movements and more time are necessary to find the target (c). (Figs 4a and 4b courtesy of Elizabeth Krupinski, PhD, University of Arizona College of Medicine, Tucson, Ariz.)

Figure 4d

Scan paths in visual search. (a, b) Scan paths (red lines) for a first-year resident (a) and an expert radiologist (b) while searching a chest radiograph for lung nodules. (c, d) Analogous scan paths for naive observers searching artificial scene stimuli. Scene guidance allows the target to be quickly found in a realistic scene (d). However, when the background is removed and the object positions are scrambled, more eye movements and more time are necessary to find the target (c). (Figs 4a and 4b courtesy of Elizabeth Krupinski, PhD, University of Arizona College of Medicine, Tucson, Ariz.)

Figure 5a

Experimental procedure (a) and results (b) of the study of Võ and Henderson (54). A 50-msec preview of a scene was found to significantly decrease the time needed to find the target item in subsequent viewing of the same scene by using a gaze-contingent window. RT in b = response time.

Figure 5b

Experimental procedure (a) and results (b) of the study of Võ and Henderson (54). A 50-msec preview of a scene was found to significantly decrease the time needed to find the target item in subsequent viewing of the same scene by using a gaze-contingent window. RT in b = response time.