Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Oct 10;9(11):5318-5329.
doi: 10.1364/BOE.9.005318. eCollection 2018 Nov 1.

Automatic classification of dual-modalilty, smartphone-based oral dysplasia and malignancy images using deep learning

Bofan Song  1   2 , Sumsum Sunny  3   4 , Ross D Uthoff  1 , Sanjana Patrick  5 , Amritha Suresh  3   4 , Trupti Kolur  3 , G Keerthi  6 , Afarin Anbarani  7 , Petra Wilder-Smith  7 , Moni Abraham Kuriakose  3   4 , Praveen Birur  5   6 , Jeffrey J Rodriguez  8 , Rongguang Liang  1   9
Affiliations

Automatic classification of dual-modalilty, smartphone-based oral dysplasia and malignancy images using deep learning

Bofan Song et al. Biomed Opt Express. .

Abstract

With the goal to screen high-risk populations for oral cancer in low- and middle-income countries (LMICs), we have developed a low-cost, portable, easy to use smartphone-based intraoral dual-modality imaging platform. In this paper we present an image classification approach based on autofluorescence and white light images using deep learning methods. The information from the autofluorescence and white light image pair is extracted, calculated, and fused to feed the deep learning neural networks. We have investigated and compared the performance of different convolutional neural networks, transfer learning, and several regularization techniques for oral cancer classification. Our experimental results demonstrate the effectiveness of deep learning methods in classifying dual-modal images for oral cancer detection.

PubMed Disclaimer

Conflict of interest statement

The authors declare that there are no conflicts of interest related to this article.

Figures

Fig. 1
Fig. 1
Low-cost, dual-modality smartphone-based oral cancer screening platform. The platform consists of a commercial LG G4 Android smartphone, an external peripheral with intraoral imaging attachment, LEDs, LED driver and batteries, a cloud-based image process and storage server. A custom Android application provides a user interface, controls the phone along with external peripherals, and enables communication with the cloud server and remote specialist.
Fig. 2
Fig. 2
Workflow of the proposed mobile imaging platform. Autofluorescence and white light images acquired from smartphone are uploaded to cloud server and classified for oral dysplasia and malignancy based on deep learning. Remote diagnosis could be provided by remote specialists anywhere with internet connections. Results can be viewed on-site through the customized mobile app.
Fig. 3
Fig. 3
Examples of dual-modal image pairs captured from the dual-modal mobile imaging device. (a) and (b) are autofluorescence image and white light image from the palate of a healthy patient, (c) and (d) are from the buccal mucosa of a patient with oral potentially malignant lesion, and (e) and (f) are image pairs of a malignant lesion from the lower vestibule.
Fig. 4
Fig. 4
Overview of the data preparation for dual-modal image classification. A new, three-channel data set is created from the autofluorescence and white light image pairs. The blue channel of the white light image which has low signal and high noise is excluded. The new three-channel image uses the green and red channels from the white light image and the normalized ratio of red and green channels from autofluorescence image as the third channel.
Fig. 5
Fig. 5
Data augmentation. Example of the data-augmented images. A total of 8 images are obtained from a single image by rotating and flipping the original image.
Fig. 6
Fig. 6
(a) The validation errors of three different neural network architectures: VGG-CNN-M, VGG-CNN-S, and VGG-16. VGG-CNN-M performs best among these three networks. (b) The comparison between the neural networks trained with and without augmented data. (c) The performances for different weight decays. (d) The performance with and without Dropout.
Fig. 7
Fig. 7
The performance of neural networks trained with dual-modal images, AFI, and WLI.
See this image and copyright information in PMC

References

    1. World Health Organization , “Oral cancer,” http://www.who.int/cancer/prevention/diagnosis-screening/oral-cancer/en/.
    1. Petti S., Scully C., “Oral cancer knowledge and awareness: primary and secondary effects of an information leaflet,” Oral Oncol. 43(4), 408–415 (2007).10.1016/j.oraloncology.2006.04.010 - DOI - PubMed
    1. American Society of Clinical Oncology , “Oral and Oropharyngeal Cancer: Statistics,” https://www.cancer.net/cancer-types/oral-and-oropharyngeal-cancer/statis....
    1. Mallath M. K., Taylor D. G., Badwe R. A., Rath G. K., Shanta V., Pramesh C. S., Digumarti R., Sebastian P., Borthakur B. B., Kalwar A., Kapoor S., Kumar S., Gill J. L., Kuriakose M. A., Malhotra H., Sharma S. C., Shukla S., Viswanath L., Chacko R. T., Pautu J. L., Reddy K. S., Sharma K. S., Purushotham A. D., Sullivan R., “The growing burden of cancer in India: epidemiology and social context,” Lancet Oncol. 15(6), e205–e212 (2014).10.1016/S1470-2045(14)70115-9 - DOI - PubMed
    1. National Institute of Cancer Prevention and Research , “Statistics,” http://cancerindia.org.in/statistics/.