Primary histologic diagnosis using automated whole slide imaging: a validation study

Abstract

Background: Only prototypes 5 years ago, high-speed, automated whole slide imaging (WSI) systems (also called digital slide systems, virtual microscopes or wide field imagers) are becoming increasingly capable and robust. Modern devices can capture a slide in 5 minutes at spatial sampling periods of less than 0.5 micron/pixel. The capacity to rapidly digitize large numbers of slides should eventually have a profound, positive impact on pathology. It is important, however, that pathologists validate these systems during development, not only to identify their limitations but to guide their evolution.

Methods: Three pathologists fully signed out 25 cases representing 31 parts. The laboratory information system was used to simulate real-world sign-out conditions including entering a full diagnostic field and comment (when appropriate) and ordering special stains and recuts. For each case, discrepancies between diagnoses were documented by committee and a "consensus" report was formed and then compared with the microscope-based, sign-out report from the clinical archive.

Results: In 17 of 25 cases there were no discrepancies between the individual study pathologist reports. In 8 of the remaining cases, there were 12 discrepancies, including 3 in which image quality could be at least partially implicated. When the WSI consensus diagnoses were compared with the original sign-out diagnoses, no significant discrepancies were found. Full text of the pathologist reports, the WSI consensus diagnoses, and the original sign-out diagnoses are available as an attachment to this publication.

Conclusion: The results indicated that the image information contained in current whole slide images is sufficient for pathologists to make reliable diagnostic decisions and compose complex diagnostic reports. This is a very positive result; however, this does not mean that WSI is as good as a microscope. Virtually every slide had focal areas in which image quality (focus and dynamic range) was less than perfect. In some cases, there was evidence of over-compression and regions made "soft" by less than perfect focus. We expect systems will continue to get better, image quality and speed will continue to improve, but that further validation studies will be needed to guide development of this promising technology.

Figure 1

Case 54, dysplastic nevus. Case 54 included a dysplastic nevus and neurotized dermal nevus. Figure 1 documents the dysplastic component at medium resolution.

Figure 2

Case 54, neurotized dermal nevus. Case 54 included a dysplastic nevus and neurotized dermal nevus. Figure 2 documents the dermal nevus at medium resolution.

Figure 3

Case 54, Tangential cut called an actinic keratosis. Case 54 included a dysplastic nevus and neurotized dermal nevus. One pathologist also reported an AK at the edge of one of the sections. Consensus was that the "AK" was actually a tangential cut at edge of section. See Figure 4 for higher power view.

Figure 4

Case 54, Tangential cut called an actinic keratosis. Case 54 included a dysplastic nevus and neurotized dermal nevus. One pathologist also reported an AK at the edge of one of the sections. Consensus was that "AK" was actually a tangential cut at edge of section.

Figure 5

Case 52, seborrheic keratosis. Case 52 was a seborrheic keratosis. One study pathologist diagnosed actinic keratosis citing the mild atypia in the epithelium. Consensus was that the correct diagnosis was seborrheic keratosis. Figure 6 shows a higher power including some of the atypical cells in the epithelium.

Figure 6

Case 52, seborrheic keratosis. Case 52 was a seborrheic keratosis. One study pathologist diagnosed actinic keratosis citing the mild atypia in the epithelium. Consensus was that the correct diagnosis was seborrheic keratosis. The atypia may have been secondary to human papilloma virus (HPV). Figure 5 shows a lower power view of a tissue section.

Figure 7

Case 38, granulomatous inflammation. A clear granuloma missed by two pathologists as they steadfastly searched for neoplasia in the epithelium

Figure 8

Case 40, bladder biopsy with inflammation. Intense inflammation made evaluation of superficial invasion difficult, even under the microscope. Figures 9 and 10 show higher power views.

Figure 9

Case 40, bladder biopsy with inflammation. Intense inflammation made evaluation of superficial invasion difficult, even under the microscope. Much of the tumor appeared low grade. Figure 8 shows a lower power view.

Figure 10

Case 40, bladder biopsy with inflammation. In several foci, the tumor cells seemed to loose polarity and had a high nuclear/cytoplasmic ration. Figure 8 shows a lower power view.

Figure 11

Case 37, bladder biopsy with urothelial carcinoma. An overview of the specimen, the diagnostic question was high grade versus low grade. Figures 12 and 13 show higher powers

Figure 12

Case 37, bladder biopsy with urothelial carcinoma. Medium view of a tissue fragment, the diagnostic question was high grade versus low grade. Figures 11 and 13 show additional views

Figure 13

Case 37, bladder biopsy with urothelial carcinoma. High power view of a tissue fragment, the diagnostic question was high grade versus low grade. Figures 11 and 12 show additional views

Figure 14

Case 38, bladder biopsy with urothelial carcinoma. High power view of the epithelium. The field is slightly out of focus, the epithelial cells are dark and one cannot appreciate and detail in their nuclei. Umbrella cells are clearly visible the upper part of image, but are not present in the center and lower areas. Without umbrella cells, it might be difficult to distinguish between a benign or dysplastic epithelium. There are at least to image problems present: focus and dynamic range (lack of detail in the dark nuclei). Though nuclei can be dark for reasons not related to images (i.e. histologic staining), in this case better nuclear detail was appreciated under the microscope.

Figure 15

Case 50, dysplastic nevus. Less than perfect imaging contributed to confusion in this case. While all pathologists felt that the lesion was a dysplastic nevus, two pathologists could not completely rule out melanoma on H&E stains.

Figure 16

Case 50, dysplastic nevus. Less than perfect imaging contributed to confusion in the case. In this field, numerous epithelial melanocytes caused concern for one pathologists that that was magnified because foci were slightly out of focus (i.e. upper left corner).

Figure 17

Case 50, dysplastic nevus. Less than perfect imaging contributed to confusion in the case. This field shows dermal melanocytes that were difficult to interpret. The pathologist requested immunoperoxidase stains to evaluate the lesion further.

Figure 18

Case 57, prostate biopsy. The study pathologists disagreed about this focus. Opinions ranged from benign to suspicious to outright cancer. Special stains were non-contributory. After discussion, it was agreed that cancer could not be definitively diagnosed. This turned out to be consistent with the signed report. The area is slightly out of focus. Figure 19 shows a higher power view.

Figure 19

Case 57, prostate biopsy. The study pathologists disagreed about this focus. Opinions ranged from benign to suspicious to outright cancer. Special stains were non-contributory. After discussion, it was agreed that cancer could not be definitively diagnosed. This turned out to be consistent with the signed report. The area is slightly out of focus. Figure 18 shows a lower power view.

Figure 20

Case 38 Part 1, The signed out report diagnosed "focal submucosal non-necrotizing granulomatous inflammation". The study pathologists (reading from both whole slide images and directly through a microscope) were more comfortable in reporting "focal area suggestive of granulomatous inflammation".

Figure 21

Case 52, nephrectomy for renal cell carcinoma. The study pathologists (reading from both whole slide images and directly through a microscope) gave this tumor a Fuhrman's Nuclear Grade of III/IV. The grade in the signed out report is ll/lV.

Figure 22

Perceived image quality. Each pathologist rated each whole slide image as "Excellent" ("Flawless – superb color and sharpness/focus"), "Diagnostic" ("Minimal distortion – quality is high enough to render a diagnosis") or "Poor" ("Extreme distortion – quality makes diagnosis difficult or impossible"). No whole slide image was rated "poor" by more than one pathologist. Six of seven times, the reason for a "poor" rating was that "key areas were out of focus".

Figure 23

Perceived case complexity. Each pathologist rated each case for "Complexity". Complexity involved aspects of diagnostic difficulty, case management (i.e. the need for recuts or special stains), and reporting issues (i.e. the need for a diagnostic comment). The study appeared to include a balanced set of cases with "high", "medium", and "low" complexity.

Figure 24

Diagnostic confidence. For each case, the pathologists rated their confidence in their diagnosis. Despite the relative novelty of WSI, diagnostic confidence was high. Out of 72 responses, there were only 3 instances of "low" diagnostic confidence (involving cases 30, 35 and 51). These cases did not involve instances of "poor" image quality and did not result in diagnostic discrepancies between pathologists.

Figure 25

System performance. Ratings provided an impression of pathologists' satisfaction with system performance (stability, response times and ease of use). The data is not quantitatively significant, but system performance was felt to be excellent to good in most cases.

Figure 26

Time to case completion. Though useful, this data was very hard to interpret as it included not only "monitor time" but also time required to write the report, order stains and, in some cases, work directly with histology lab to identify and orient blocks.

Figure 27

Case 57, poorly imaged area of adenocarcinoma. This image shows several image problems. The field was poorly focused and the nuclei appear smeared. Because the rest of the case had reasonably good imaging (including other levels of the same lesion), the pathologists could easily read around the artifact and arrived at the correct diagnosis.