Machine learning techniques for mitoses classification

Abstract

Background: Pathologists analyze biopsy material at both the cellular and structural level to determine diagnosis and cancer stage. Mitotic figures are surrogate biomarkers of cellular proliferation that can provide prognostic information; thus, their precise detection is an important factor for clinical care. Convolutional Neural Networks (CNNs) have shown remarkable performance on several recognition tasks. Utilizing CNNs for mitosis classification may aid pathologists to improve the detection accuracy.

Methods: We studied two state-of-the-art CNN-based models, ESPNet and DenseNet, for mitosis classification on six whole slide images of skin biopsies and compared their quantitative performance in terms of sensitivity, specificity, and F-score. We used raw RGB images of mitosis and non-mitosis samples with their corresponding labels as training input. In order to compare with other work, we studied the performance of these classifiers and two other architectures, ResNet and ShuffleNet, on the publicly available MITOS breast biopsy dataset and compared the performance of all four in terms of precision, recall, and F-score (which are standard for this data set), architecture, training time and inference time.

Results: The ESPNet and DenseNet results on our primary melanoma dataset had a sensitivity of 0.976 and 0.968, and a specificity of 0.987 and 0.995, respectively, with F-scores of .968 and .976, respectively. On the MITOS dataset, ESPNet and DenseNet showed a sensitivity of 0.866 and 0.916, and a specificity of 0.973 and 0.980, respectively. The MITOS results using DenseNet had a precision of 0.939, recall of 0.916, and F-score of 0.927. The best published result on MITOS (Saha et al. 2018) reported precision of 0.92, recall of 0.88, and F-score of 0.90. In our architecture comparisons on MITOS, we found that DenseNet beats the others in terms of F-Score (DenseNet 0.927, ESPNet 0.890, ResNet 0.865, ShuffleNet 0.847) and especially Recall (DenseNet 0.916, ESPNet 0.866, ResNet 0.807, ShuffleNet 0.753), while ResNet and ESPNet have much faster inference times (ResNet 6 s, ESPNet 8 s, DenseNet 31 s). ResNet is faster than ESPNet, but ESPNet has a higher F-Score and Recall than ResNet, making it a good compromise solution.

Conclusion: We studied several state-of-the-art CNNs for detecting mitotic figures in whole slide biopsy images. We evaluated two CNNs on a melanoma cancer dataset and then compared four CNNs on a public breast cancer data set, using the same methodology on both. Our methodology and architecture for mitosis finding in both melanoma and breast cancer whole slide images has been thoroughly tested and is likely to be useful for finding mitoses in any whole slide biopsy images.

Keywords: Convolutional neural networks; Machine learning; Melanoma; Mitoses; Pathology.

Figure 1

Example crops of biopsy images with mitoses in them; (top) skin; (bottom) breast. These biopsies are different in terms of color, texture, and mitosis phase and shape. *A mitosis in each image is present near the center and is marked with a green circle for visualization.

Figure 2

Examples of applying the nuclei segmentation method [28] on a crop of skin biopsy image (a) original crop (b) nuclei segmentation result * Two mitoses that are present in the original crop are marked with red dots for visualization. * Segmentation method was able to find the mitoses. We marked them here with red boxes for visualization.

Figure 3

Examples of (top) sampled mitoses, and (bottom) sampled nuclei that are not mitoses. These two entities have similarity in color, surrounding and texture.

Figure 4

Two convolutional units, ESPNet (a) and DenseNet (b), that are used in our experiment. Each of these units receives a 3D tensor with width W, height H, and depth N as an input and produces a 3D tensor with width W, height H, and depth M as an output. The projection channel dimension in ESPNet unit is represented by d while in DenseNet unit, it is represented by d_i. For ESPNet, output tensor depth is M = k × d, where k is the number of parallel branches in the ESPNet unit (k = 3 in (a)), the size of the point-wise convolution is 1 × 1, and n_i is the size of the dilated convolutional layers. For more information, see [33]. For the DenseNet unit, output tensor depth is M = ∑d_i, i = {1, … , L}, where L represents the number of stacked layers (L = 3 in (b)). It is common to use 3 × 3 standard convolutional layers in DenseNets. For more information, see [34].