Advancement and independent validation of a deep learning-based tool for automated scoring of nail psoriasis severity using the modified nail psoriasis severity index

Abstract

Objective: To improve and validate a convolutional neural network (CNN)-based model for the automated scoring of nail psoriasis severity using the modified Nail Psoriasis Severity Index (mNAPSI) with adequate accuracy across all severity classes and without dependency on standardized conditions.

Methods: Patients with psoriasis (PsO), psoriatic arthritis (PsA), and non-psoriatic controls including healthy individuals and patients with rheumatoid arthritis were included for training, while validation utilized an independent cohort of psoriatic patients. Nail photographs were pre-processed and segmented and mNAPSI scores were annotated by five expert readers. A CNN based on Bidirectional Encoder representation from Image Transformers (BEiT) architecture and pre-trained on ImageNet-22k was fine-tuned for mNAPSI classification. Model performance was compared with human annotations by using area under the receiver operating characteristic curve (AUROC) and other metrics. A reader study was performed to assess inter-rater variability.

Results: In total, 460 patients providing 4,400 nail photographs were included in the training dataset. The independent validation dataset included 118 further patients who provided 929 nail photographs. The CNN demonstrated high classification performance on the training dataset, achieving mean (SD) AUROC of 86% ± 7% across mNAPSI classes. Performance remained robust on the independent validation dataset, with a mean AUROC of 80% ± 9%, despite variability in imaging conditions. Compared with human annotation, the CNN achieved a Pearson correlation of 0.94 on a patient-level, which remained consistent in the validation dataset.

Conclusion: We developed and validated a CNN that enables the automated, objective scoring of nail psoriasis severity based on mNAPSI with high reliability and without need of image standardization. This approach has potential clinical utility for enabling a standardized time-efficient assessment of nail involvement in the psoriatic disease and possibly as a self-reporting tool.

Keywords: MNAPSI; NAPSI; artificial intelligence; machine learning; nail disease; outcome measures; psoriasis; psoriatic arthritis.

Figure 1

Pipeline of the data collection (a) and of the model training process (b). (c) Shows visual examples of nail changes throughout the different mNAPSI classes 1–5.

Figure 2

Histograms of the mNAPSI annotation of collected nails.

Figure 3

Least-squares fit analyses of human annotation against CNN-predicted mNAPSI at patient-level for training (left panel) and validation datasets (right panel).

Figure 4

Confusion matrix of the validation dataset (right) and test data from training dataset (left).

Figure 5

ROC (left) and PRC (right) curves of the training (a) and validation (b) dataset.

Figure 6

Influence of training set sizes on the performance of the neural network.