Federated Learning for Decentralized Artificial Intelligence in Melanoma Diagnostics

Abstract

Importance: The development of artificial intelligence (AI)-based melanoma classifiers typically calls for large, centralized datasets, requiring hospitals to give away their patient data, which raises serious privacy concerns. To address this concern, decentralized federated learning has been proposed, where classifier development is distributed across hospitals.

Objective: To investigate whether a more privacy-preserving federated learning approach can achieve comparable diagnostic performance to a classical centralized (ie, single-model) and ensemble learning approach for AI-based melanoma diagnostics.

Design, setting, and participants: This multicentric, single-arm diagnostic study developed a federated model for melanoma-nevus classification using histopathological whole-slide images prospectively acquired at 6 German university hospitals between April 2021 and February 2023 and benchmarked it using both a holdout and an external test dataset. Data analysis was performed from February to April 2023.

Exposures: All whole-slide images were retrospectively analyzed by an AI-based classifier without influencing routine clinical care.

Main outcomes and measures: The area under the receiver operating characteristic curve (AUROC) served as the primary end point for evaluating the diagnostic performance. Secondary end points included balanced accuracy, sensitivity, and specificity.

Results: The study included 1025 whole-slide images of clinically melanoma-suspicious skin lesions from 923 patients, consisting of 388 histopathologically confirmed invasive melanomas and 637 nevi. The median (range) age at diagnosis was 58 (18-95) years for the training set, 57 (18-93) years for the holdout test dataset, and 61 (18-95) years for the external test dataset; the median (range) Breslow thickness was 0.70 (0.10-34.00) mm, 0.70 (0.20-14.40) mm, and 0.80 (0.30-20.00) mm, respectively. The federated approach (0.8579; 95% CI, 0.7693-0.9299) performed significantly worse than the classical centralized approach (0.9024; 95% CI, 0.8379-0.9565) in terms of AUROC on a holdout test dataset (pairwise Wilcoxon signed-rank, P < .001) but performed significantly better (0.9126; 95% CI, 0.8810-0.9412) than the classical centralized approach (0.9045; 95% CI, 0.8701-0.9331) on an external test dataset (pairwise Wilcoxon signed-rank, P < .001). Notably, the federated approach performed significantly worse than the ensemble approach on both the holdout (0.8867; 95% CI, 0.8103-0.9481) and external test dataset (0.9227; 95% CI, 0.8941-0.9479).

Conclusions and relevance: The findings of this diagnostic study suggest that federated learning is a viable approach for the binary classification of invasive melanomas and nevi on a clinically representative distributed dataset. Federated learning can improve privacy protection in AI-based melanoma diagnostics while simultaneously promoting collaboration across institutions and countries. Moreover, it may have the potential to be extended to other image classification tasks in digital cancer histopathology and beyond.

Figure 1.. Flowchart of the Slide Inclusion Process

Slides were excluded from the analysis if there was no histopathologically confirmed label available or if the lesion proved to be neither invasive melanoma (IM) nor nevus (in situ tumors or other diagnoses, eg, basal cell carcinoma, squamous cell carcinoma). In addition, slides that exhibited fewer than 50 epidermal patches or other technical issues were removed.

Figure 2.. Mean Area Under the Receiver Operating Characteristic Curve (AUROC) of the 3 Investigated Approaches

Mean AUROCs on the holdout and external test dataset after 1000 iterations of bootstrapping, including the corresponding 95% CIs (shaded areas), are illustrated for the federated learning (FL) and the centralized approach (model Hfull) (A and B) and for the FL and the ensemble approach (C and D). AUC indicates area under the curve.