Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Oct:156:202-216.
doi: 10.1016/j.ejca.2021.06.049. Epub 2021 Sep 8.

Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts

Sarah Haggenmüller  1 , Roman C Maron  1 , Achim Hekler  1 , Jochen S Utikal  2 , Catarina Barata  3 , Raymond L Barnhill  4 , Helmut Beltraminelli  5 , Carola Berking  6 , Brigid Betz-Stablein  7 , Andreas Blum  8 , Stephan A Braun  9 , Richard Carr  10 , Marc Combalia  11 , Maria-Teresa Fernandez-Figueras  12 , Gerardo Ferrara  13 , Sylvie Fraitag  14 , Lars E French  15 , Frank F Gellrich  16 , Kamran Ghoreschi  17 , Matthias Goebeler  18 , Pascale Guitera  19 , Holger A Haenssle  20 , Sebastian Haferkamp  21 , Lucie Heinzerling  22 , Markus V Heppt  6 , Franz J Hilke  17 , Sarah Hobelsberger  16 , Dieter Krahl  23 , Heinz Kutzner  24 , Aimilios Lallas  25 , Konstantinos Liopyris  26 , Mar Llamas-Velasco  27 , Josep Malvehy  11 , Friedegund Meier  16 , Cornelia S L Müller  28 , Alexander A Navarini  29 , Cristián Navarrete-Dechent  30 , Antonio Perasole  31 , Gabriela Poch  17 , Sebastian Podlipnik  11 , Luis Requena  32 , Veronica M Rotemberg  33 , Andrea Saggini  24 , Omar P Sangueza  34 , Carlos Santonja  35 , Dirk Schadendorf  36 , Bastian Schilling  18 , Max Schlaak  17 , Justin G Schlager  22 , Mildred Sergon  16 , Wiebke Sondermann  37 , H Peter Soyer  7 , Hans Starz  38 , Wilhelm Stolz  39 , Esmeralda Vale  40 , Wolfgang Weyers  41 , Alexander Zink  42 , Eva Krieghoff-Henning  1 , Jakob N Kather  43 , Christof von Kalle  44 , Daniel B Lipka  45 , Stefan Fröhling  45 , Axel Hauschild  46 , Harald Kittler  47 , Titus J Brinker  48
Affiliations
Free article

Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts

Sarah Haggenmüller et al. Eur J Cancer. 2021 Oct.
Free article

Abstract

Background: Multiple studies have compared the performance of artificial intelligence (AI)-based models for automated skin cancer classification to human experts, thus setting the cornerstone for a successful translation of AI-based tools into clinicopathological practice.

Objective: The objective of the study was to systematically analyse the current state of research on reader studies involving melanoma and to assess their potential clinical relevance by evaluating three main aspects: test set characteristics (holdout/out-of-distribution data set, composition), test setting (experimental/clinical, inclusion of metadata) and representativeness of participating clinicians.

Methods: PubMed, Medline and ScienceDirect were screened for peer-reviewed studies published between 2017 and 2021 and dealing with AI-based skin cancer classification involving melanoma. The search terms skin cancer classification, deep learning, convolutional neural network (CNN), melanoma (detection), digital biomarkers, histopathology and whole slide imaging were combined. Based on the search results, only studies that considered direct comparison of AI results with clinicians and had a diagnostic classification as their main objective were included.

Results: A total of 19 reader studies fulfilled the inclusion criteria. Of these, 11 CNN-based approaches addressed the classification of dermoscopic images; 6 concentrated on the classification of clinical images, whereas 2 dermatopathological studies utilised digitised histopathological whole slide images.

Conclusions: All 19 included studies demonstrated superior or at least equivalent performance of CNN-based classifiers compared with clinicians. However, almost all studies were conducted in highly artificial settings based exclusively on single images of the suspicious lesions. Moreover, test sets mainly consisted of holdout images and did not represent the full range of patient populations and melanoma subtypes encountered in clinical practice.

Keywords: Artificial intelligence; Convolutional neural network(s); Deep learning; Dermatology; Digital biomarkers; Machine learning; Malignant melanoma; Skin cancer classification.

PubMed Disclaimer

Conflict of interest statement

Conflict of interest statement The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: J.S.U. is on the advisory board or has received honoraria and travel support from Amgen, Bristol Myers Squibb, GSK, LEO Pharma, Merck Sharp and Dohme, Novartis, Pierre Fabre and Roche, outside the submitted work. M.G. has received speaker's honoraria and/or has served as a consultant and/or member of advisory boards for Almirall, Argenx, Biotest, Eli Lilly, Janssen Cilag, LEO Pharma, Novartis and UCB, outside the submitted work. H.A.H. worked as a consultant or received honoraria and travel support from Heine Optotechnik GmbH, JenLab GmbH, FotoFinder Systems GmbH, Magnosco GmbH, SciBase AB, Beiersdorf AG, Almirall Hermal GmbH and Galderma Laboratorium GmbH. V.M.R. is on the advisory board or has received honoraria or ownership in Inhabit Brands, Inc. unrelated to this work. Sondermann W. reports grants from medi GmbH Bayreuth, personal fees from Janssen, grants and personal fees from Novartis, personal fees from Lilly, personal fees from UCB, personal fees from Almirall, personal fees from LEO Pharma and personal fees from Sanofi Genzyme, outside the submitted work. H.P.S. is a shareholder of MoleMap NZ Limited and e-derm consult GmbH and undertakes regular tele-dermatological reporting for both companies. H.P.S. is a medical consultant for Canfield Scientific, Inc., MoleMap Australia Pty Ltd and Revenio Research Oy and a medical advisor for First Derm. M.L-V. has received speaker's honoraria and/or received grants and/or participated in clinical trials of AbbVie, Almirall, Amgen, Celgene, Eli Lilly, Janssen Cilag, LEO Pharma, Novartis and UCB, outside the submitted work. A.Z. has been an advisor and/or received speaker's honoraria and/or received grants and/or participated in clinical trials of AbbVie, Almirall, Amgen, Beiersdorf Dermo Medical, Bencard Allergy, Celgene, Eli Lilly, Janssen Cilag, LEO Pharma, Novartis, Sanofi-Aventis and UCB Pharma, outside the submitted work. Kittler H. received speaker's honoraria from FotoFinder Systems GmbH and received non-financial support from Heine Optotechnik GmbH, Derma Medical and 3Gen. T.J.B. reports owning a company that develops mobile apps, including the teledermatology services AppDoc (https://online-hautarzt.de) and Intimarzt (https://Intimarzt.de); Smart Health Heidelberg GmbH, Handschuhsheimer Landstr. 9/1, 69120 Heidelberg, https://smarthealth.de. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publication types

LinkOut - more resources