Pigeons (Columba livia) as Trainable Observers of Pathology and Radiology Breast Cancer Images

Abstract

Pathologists and radiologists spend years acquiring and refining their medically essential visual skills, so it is of considerable interest to understand how this process actually unfolds and what image features and properties are critical for accurate diagnostic performance. Key insights into human behavioral tasks can often be obtained by using appropriate animal models. We report here that pigeons (Columba livia)-which share many visual system properties with humans-can serve as promising surrogate observers of medical images, a capability not previously documented. The birds proved to have a remarkable ability to distinguish benign from malignant human breast histopathology after training with differential food reinforcement; even more importantly, the pigeons were able to generalize what they had learned when confronted with novel image sets. The birds' histological accuracy, like that of humans, was modestly affected by the presence or absence of color as well as by degrees of image compression, but these impacts could be ameliorated with further training. Turning to radiology, the birds proved to be similarly capable of detecting cancer-relevant microcalcifications on mammogram images. However, when given a different (and for humans quite difficult) task-namely, classification of suspicious mammographic densities (masses)-the pigeons proved to be capable only of image memorization and were unable to successfully generalize when shown novel examples. The birds' successes and difficulties suggest that pigeons are well-suited to help us better understand human medical image perception, and may also prove useful in performance assessment and development of medical imaging hardware, image processing, and image analysis tools.

Fig 1. The pigeons’ training environment.

The operant conditioning chamber was equipped with a food pellet dispenser, and a touch-sensitive screen upon which the medical image (center) and choice buttons (blue and yellow rectangles) were presented.

Fig 2. Examples of benign (left) and malignant (right) breast specimens stained with hematoxylin and eosin, at different magnifications.

Pigeons were initially trained and tested with samples at 4× magnification (top row), and then were subsequently transitioned to samples at 10× magnification (center row) and 20× magnification (bottom row).

Fig 3. Monochrome images with equated hue and brightness, at different levels of compression.

The original images at 10× magnification were converted to grayscale, colored with a single hue, and had their overall brightness and contrast equalized as closely as possible. Additionally, the images were reduced to 7% (1:15, middle row) or 4% (1:27, bottom row) of their original size, to create the compressed sets.

Fig 4. Mammograms with the absence (left) and with presence (right) of microcalcifications.

Yellow circles denote where microcalcifications are located.

Fig 5. Examples of benign (left) and malignant (right) masses in mammograms.

Subsequent biopsy established histopathology ground-truth.

Fig 6. Results of training with breast histopathology samples at different magnifications and rotations.

A) When first trained with 4× magnification images the birds performed at chance levels of accuracy, but quickly learned to discriminate. Subsequently, when the birds were exposed to higher magnifications samples, their performance commenced at accuracies above chance (but below their final performance at lower magnification they had previously been exposed to), and improved further with training. B) Introducing rotated versions of the training stimuli did not significantly affect performance at any of the magnifications.

Fig 7. Generalization from training to test image sets.

After training with differential reinforcement, the birds successfully classified previously unseen breast tissue images in the testing sets, at all magnifications, with no statistically significant decrease in accuracy compared to training-set performance.

Fig 8. Training and testing with hue- and brightness-normalized breast histology images.

A) The pigeons were able to learn discrimination without the benefit of hue and brightness cues. B) However, the lack of these cues diminished the birds’ ability to generalize to new images; compared to an equivalent test of full-color exemplars (see Fig 7), the pigeons performed significantly more poorly, although still well above chance levels.

Fig 9. Flock sourcing.

A “flock-sourcing” score was calculated by summating the responses of individual birds as described in the text. Pooling the birds’ decisions led to significantly better discrimination than that achieved by individual pigeons. The dotted line represents no discrimination between benign and malignant exemplars.

Fig 10. Effect of JPEG image compression.

When correct/incorrect responses were nondifferentially reinforced (gray bars), pigeons’ accuracy was affected proportionally to the compression level of the images shown. However, pigeons were capable of achieving high levels of accuracy with compressed images if feedback for correct/incorrect responses was given (white bars).

Fig 11. Results of training and testing with mammograms with or without calcifications.

A) Training quickly led to high levels of accuracy. B) The pigeons were able to generalize to novel images, but their performance on this task was not as good as their generalization to novel histology images (Fig 7), although still above chance levels of responding. The trend observed on Day 1 of testing (left) continued throughout the remainder of testing (right).

Fig 12. Results of training and testing with mammograms containing masses.

A) Pigeons required long training to discriminate between mammograms with masses, and even then, individual differences were pronounced. B) Regardless of their performance in the training phase, all of the pigeons failed to transfer their performance to novel exemplars, suggesting that their performance was based on rote memorization.

Fig 13. Conflictive histology exemplars.

During Experiment 1, some exemplars from a given category looked like exemplars from the other category causing the birds to incorrectly categorize them.