Variability in physician assessment of lesions in cutaneous images and its implications for skin screening and computer-assisted diagnosis

Abstract

Background and design: The diagnosis of skin lesions is a three-part process of (1) screening for abnormalities, (2) examining lesions closely for the presence of important clinical characteristics, and (3) choosing a diagnosis based on the clinical data gathered. The first step, screening for abnormal areas, is of particular interest for anyone wishing to understand and automate the diagnostic process. As part of ongoing research into lesion detection and measurement, we examined the reproducibility and reliability of dermatologists who were asked to screen sets of cutaneous digital images and high-resolution photographs for the presence of skin lesions.

Results: The intraobserver and interobserver variation in this task was considerable. The average reproducibilities and reliabilities were approximately 85%. The photographs did not produce significantly better overall screening results than the moderately low-resolution digital images.

Conclusions: Even highly trained dermatologists appear to differ significantly in their initial interpretations of skin images, both from observer to observer and within observers from session to session. These differences were reflected in all measures of diagnostic sensitivity, reproducibility, and reliability assessed. The magnitude of this variation is relatively insensitive to the means used to measure it, and is not simply due to different individual standards for diagnosis. It is highly unlikely that this variation is attributable to the images themselves, since the overall results were nearly identical for moderately low-resolution digital formats and very high-resolution photographs. These findings have important implications for determining where errors may take place in dermatologic screening and for automating the process of detecting changes in skin lesions over time.