Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Mar;34(3):648-655.
doi: 10.1111/jdv.15935. Epub 2019 Oct 8.

Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms

A Udrea  1   2 , G D Mitra  2 , D Costea  1   2 , E C Noels  3 , M Wakkee  3 , D M Siegel  4   5 , T M de Carvalho  3 , T E C Nijsten  3
Affiliations

Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms

A Udrea et al. J Eur Acad Dermatol Venereol. 2020 Mar.

Abstract

Background: Machine learning algorithms achieve expert-level accuracy in skin lesion classification based on clinical images. However, it is not yet shown whether these algorithms could have high accuracy when embedded in a smartphone app, where image quality is lower and there is high variability in image taking scenarios by users. In the past, these applications were criticized due to lack of accuracy.

Objective: In this study, we evaluate the accuracy of the newest version of a smartphone application (SA) for risk assessment of skin lesions.

Methods: This SA uses a machine learning algorithm to compute a risk rating. The algorithm is trained on 131 873 images taken by 31 449 users in multiple countries between January 2016 and August 2018 and rated for risk by dermatologists. To evaluate the sensitivity of the algorithm, we use 285 histopathologically validated skin cancer cases (including 138 malignant melanomas), from two previously published clinical studies (195 cases) and from the SA user database (90 cases). We calculate the specificity on a separate set from the SA user database containing 6000 clinically validated benign cases.

Results: The algorithm scored a 95.1% (95% CI, 91.9-97.3%) sensitivity in detecting (pre)malignant conditions (93% for malignant melanoma and 97% for keratinocyte carcinomas and precursors). This level of sensitivity was achieved with a 78.3% (95% CI, 77.2-79.3%) specificity.

Conclusions: This SA provides a high sensitivity to detect skin cancer; however, there is still room for improvement in terms of specificity. Future studies are needed to assess the impact of this SA on the health systems and its users.

PubMed Disclaimer

References

    1. Ngoo A, Finnane A, McMeniman E, Soyer HP, Janda M. Fighting melanoma with smartphones: a snapshot of where we are a decade after app stores opened their doors. Int J Med Informatics 2018; 118: 99-112.
    1. Codella NCF, Gutman D, Celebi ME et al. Skin lesion analysis toward melanoma detection: a challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), hosted by the International Skin Imaging Collaboration (ISIC). arXiv, 1710.05006v3 [cs.CV]
    1. Esteva A, Kuprel B, Novoa RA et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542: 115-118.
    1. Haenssle HA, Fink C, Schneiderbauer R et al. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol 2018; 29: 1836-1842.
    1. SEER. Cancer stat facts: melanoma of the skin. https://seer.cancer.gov/statfacts/html/melan.html (last accessed: 24 August 2018).

LinkOut - more resources