Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jan 17;16(1):789.
doi: 10.1038/s41467-025-56160-x.

Discordance, accuracy and reproducibility study of pathologists' diagnosis of melanoma and melanocytic tumors

Sarah Haggenmüller  1 , Christoph Wies  1   2 , Julia Abels  1 , Jana T Winterstein  1   2 , Lukas Heinlein  1   2 , Carina Nogueira Garcia  1 , Jochen S Utikal  3   4   5 , Sebastian A Wohlfeil  3   4   5 , Friedegund Meier  6 , Sarah Hobelsberger  6 , Frank F Gellrich  6 , Mildred Sergon  7   8 , Axel Hauschild  9 , Lars E French  10   11 , Lucie Heinzerling  10   12 , Justin G Schlager  10 , Kamran Ghoreschi  13 , Max Schlaak  13 , Franz J Hilke  13 , Gabriela Poch  13 , Sören Korsing  13 , Cosimo Sarfert  13 , Carola Berking  12 , Markus V Heppt  12 , Michael Erdmann  12 , Sebastian Haferkamp  14 , Konstantin Drexler  14 , Dirk Schadendorf  15 , Wiebke Sondermann  15 , Matthias Goebeler  16 , Bastian Schilling  17 , Jakob Nikolas Kather  18 , Stefan Fröhling  19 , Mar Llamas-Velasco  20 , Luis C Requena  21 , Gerardo Ferrara  22 , Maite Fernandez-Figueras  23 , Sylvie Fraitag  24 , Cornelia S L Müller  25   26 , Hans Starz  27 , Heinz Kutzner  28 , Raymond Barnhill  29 , Richard Carr  30 , Kenneth S Resnik  31 , Stephan Alexander Braun  32 , Tim Holland-Letz  33 , Titus J Brinker  34
Affiliations

Discordance, accuracy and reproducibility study of pathologists' diagnosis of melanoma and melanocytic tumors

Sarah Haggenmüller et al. Nat Commun. .

Abstract

Accurate melanoma diagnosis is crucial for patient outcomes and reliability of AI diagnostic tools. We assess interrater variability among eight expert pathologists reviewing histopathological images and clinical metadata of 792 melanoma-suspicious lesions prospectively collected at eight German hospitals. Moreover, we provide access to the largest panel-validated dataset featuring dermoscopic and histopathological images with metadata. Complete agreement is achieved in 53.5% of cases (424/792), and a majority vote ( ≥ five pathologists) in 90.9% (720/792). Considerable discordance is observed for non-invasive melanomas (complete agreement in only 10/73 cases). The expert panel disagrees with the local pathologists' and dermatologists' diagnoses in 14.9% and 33.5% of cases, respectively. This variability highlights the diagnostic challenges of early-stage melanomas and the need to reconsider how ground truth is established in routine care and AI research. Including at least two pathologists or virtual panels may contribute to more consistent diagnostic results.

PubMed Disclaimer

Conflict of interest statement

Competing interests: Prof Utikal reported personal fees from Amgen, Bristol Myers Squibb, GSK, Immunocore, LEO Pharma, Merck Sharp & Dohme, Novartis, Pierre Fabre, Rheacell, Roche, and Sanofi outside the submitted work. Dr Wohlfeil received honoraria from Bristol Myers Squibb, Novartis, and Sun Pharma outside the submitted work. Prof Meier reported grants from Novartis and Roche and others (travel support or/and speaker’s fees or/and advisor’s honoraria) from BMS, MSD, and Pierre Fabre outside the submitted work. Dr Hobelsberger reported clinical trial support and advisor’s honoraria from Almirall, speaker’s honoraria from Almirall, UCB, and AbbVie, and travel support from UCB, Janssen Cilag, Almirall, Novartis, Lilly, LEO Pharma, and AbbVie outside the submitted work. Prof Heinzerling reported other (clinical studies) from BMS, MSD, Pierre Fabre, Replimune, and Sanofi; personal fees from Biomedx, BMS, MSD, Sun, Pierre Fabre, Novartis, and Sanofi; and grants from Therakos outside the submitted work. Prof Haferkamp reported speaker’s fees and advisor’s honoraria from BMS, MSD, Novartis, and Pierre Fabre outside the submitted work. Dr Schlaak reported personal fees from BMS, Novartis, Immunocore, Sun Pharma, MSD, Recordati, and Sanofi Aventis outside the submitted work. Prof Berking reported personal fees from BMS, MSD, InflaRx, Novartis, Sanofi, LEO Pharma, Almirall Hermal, Pierre Fabre, Immunocore, and Delcath outside the submitted work. Dr Erdmann received travel support and speaker’s honoraria from Immunocore, Novartis, Pierre Fabre, and Sanofi outside the submitted work. Dr Sondermann reported grants from Almirall, Novartis, and Medi GmbH; and personal fees from AbbVie, Almirall, BMS, Boehringer Ingelheim, Celgene, Janssen, LEO Pharma, Lilly, Novartis, Pfizer, Sanofi Genzyme, and UCB outside the submitted work. Prof Goebeler reported grants for clinical studies from Argenx, Novartis, Janssen, and Galderma; personal fees from Almirall (consulting), Janssen (advisory board, speaker), GSK (advisory board, speaker), and Lilly (speaker) outside the submitted work. Dr Llamas-Velasco reported fees as an advisory board member, consultant, research support, participation in clinical trials, and honorary for speaking with the following pharmaceutical companies: Abbvie, Almirall, Amgen, Boehringer, BMS, Celgene, Janssen, Leo, Lilly, Kyowa kirin, Novartis and UCB, outside the submitted work. Dr. Kather declares consulting services for Owkin, France; DoMore Diagnostics, Norway; Panakeia, UK; AstraZeneca, UK; Scailyte, Switzerland; Mindpeak, Germany; and MultiplexDx, Slovakia. Furthermore, he holds shares in StratifAI GmbH, Germany, has received a research grant from GSK, and has received honoraria from AstraZeneca, Bayer, Eisai, Janssen, MSD, BMS, Roche, Pfizer, and Fresenius. Dr Gabriela Poch received travel support and speaker’s honoraria from MSD, BMS, Novartis, Sun Pharma, and Amgen outside the submitted work. Dr Brinker reported being the owner of Smart Health Heidelberg GmbH outside the submitted work. The remaining authors declare no competing interests.

References

    1. Schadendorf, D. et al. Melanoma. Lancet392, 971–984 (2018). - PubMed
    1. Arnold, M. et al. Global burden of cutaneous melanoma in 2020 and projections to 2040. JAMA Dermatol.158, 495–503 (2022). - PMC - PubMed
    1. Dzwierzynski, W. W. Melanoma risk factors and prevention. Clin. Plast. Surg.48, 543–550 (2021). - PubMed
    1. Cherobin, A. C. F. P., Wainstein, A. J. A., Colosimo, E. A., Goulart, E. M. A. & Bittencourt, F. V. Prognostic factors for metastasis in cutaneous melanoma. An. Bras. Dermatol.93, 19–26 (2018). - PMC - PubMed
    1. Jitian Mihulecea, C.-R., Frățilă, S. & Rotaru, M. Clinical-dermoscopic similarities between atypical nevi and early stage melanoma. Exp. Ther. Med.22, 854 (2021). - PMC - PubMed

LinkOut - more resources