Automatic Classification of Cancerous Tissue in Laserendomicroscopy Images of the Oral Cavity using Deep Learning

Abstract

Oral Squamous Cell Carcinoma (OSCC) is a common type of cancer of the oral epithelium. Despite their high impact on mortality, sufficient screening methods for early diagnosis of OSCC often lack accuracy and thus OSCCs are mostly diagnosed at a late stage. Early detection and accurate outline estimation of OSCCs would lead to a better curative outcome and a reduction in recurrence rates after surgical treatment. Confocal Laser Endomicroscopy (CLE) records sub-surface micro-anatomical images for in vivo cell structure analysis. Recent CLE studies showed great prospects for a reliable, real-time ultrastructural imaging of OSCC in situ. We present and evaluate a novel automatic approach for OSCC diagnosis using deep learning technologies on CLE images. The method is compared against textural feature-based machine learning approaches that represent the current state of the art. For this work, CLE image sequences (7894 images) from patients diagnosed with OSCC were obtained from 4 specific locations in the oral cavity, including the OSCC lesion. The present approach is found to outperform the state of the art in CLE image recognition with an area under the curve (AUC) of 0.96 and a mean accuracy of 88.3% (sensitivity 86.6%, specificity 90%).

Figure 1

Left: CLE recording locations. Additionally, the region of the suspected HNSCC was recorded. Right: Division of (resized) image into patches of size 80 × 80 px. Only patches that were inside the image mask and had no artifact labels within them were considered for classification.

Figure 2

Overview of the CNN-based patch extraction and classification.

Figure 3

Overview of the patch probability fusion approach. Left: Overlapping patches are extracted from the image and classified. Subsequently, the image classification is fused. Right: Examples for color-coded image probability maps.

Figure 4

Overview of the transfer learning approach, based on Szegedy’s Inception v3, pre-trained on the ImageNet database.

Figure 5

ROC curve of cross-validation. All results of the single cross-validation steps were combined into one result vector.

Figure 6

Randomly selected CNN patch (a) and image (b) classifications with overall high probability for clinically normal and carcinogenic tissue (top and bottom) and uncertain images (middle). N indicates images representing clinically normal (presumably healthy) tissue, C carcinogenic images. The probability given is the a posteriori probability for the class carcinogenic.