Intraobserver agreement in interpretation of digital epiluminescence microscopy

Abstract

Background: Although a major problem with the classification of epiluminescence microscopy (ELM) findings is the lack of standard definitions, reproducibility of the criteria proposed has never been investigated.

Objectives: Our purposes were (1) to perform a review of four major published classifications to obtain a set of apparently well-defined ELM variables and descriptors and (2) to evaluate the ability of one of us to report consistently ELM findings in melanocytic lesions according to these criteria.

Methods: Intraobserver agreement (with a set of 44 selected descriptors) between two readings of 150 digital ELM images was evaluated with the kappa (kappa) statistic. Subgroups of descriptors were compared for kappa value distribution.

Results: The median kappa value for the whole series of descriptors was 0.66. Median kappa did not vary significantly among the four classification systems (kappa = 0.61 to 0.67). Agreement was significantly better as to the presence or absence of ELM findings (kappa range, 0.39 to 1.00; median kappa, 0.77) compared with agreement as to their distribution (kappa range, 0.10 to 0.79; median kappa, 0.47; p = 0.0007) and their width, thickness, and size (kappa range, 0.06 to 0.83; median kappa, 0.39; p = 0.0075).

Conclusion: Although nothing can be inferred from a single study, descriptors associated with low intraobserver agreement are likely to be inadequately defined.