Evaluation of the Diagnostic Accuracy of an Online Artificial Intelligence Application for Skin Disease Diagnosis

Abstract

Artificial intelligence (AI) algorithms for automated classification of skin diseases are available to the consumer market. Studies of their diagnostic accuracy are rare. We assessed the diagnostic accuracy of an open-access AI application (Skin Image Search™) for recognition of skin diseases. Clinical images including tumours, infective and inflammatory skin diseases were collected at the Department of Dermatology at the Sahlgrenska University Hospital and uploaded for classification by the online application. The AI algorithm classified the images giving 5 differential diagnoses, which were then compared to the diagnoses made clinically by the dermatologists and/or histologically. We included 521 images portraying 26 diagnoses. The diagnostic accuracy was 56.4% for the top 5 suggested diagnoses and 22.8% when only considering the most probable diagnosis. The level of diagnostic accuracy varied considerably for diagnostic groups. The online application demonstrated low diagnostic accuracy compared to a dermatologist evaluation and needs further development.

Keywords: dermatology; online diagnostics; skin disease; artificial intelligence.

Fig. 1

The overall diagnostic accuracy of the Skin Image Search™ application. The scores 0–5 are colour-coded by different shades of blue with darker shades portraying more accurate scores. The artificial intelligence application was able to place the correct diagnosis among the top 5 in 56.4% (294/521) of the images and failed to give the correct diagnosis in 43.6% (227/521) of the images. CI: confidence interval.

Fig. 2

Diagnostic accuracy for the diagnostic groups.

Fig. 3

Diagnostic accuracy for individual diagnoses. The bars are encoded with different colours depending on the given scores (0–5). Diagnoses with <10 images uploaded for classification are presented in parentheses.