The Promise of Semantic Segmentation in Detecting Actinic Keratosis Using Clinical Photography in the Wild

Abstract

AK is a common precancerous skin condition that requires effective detection and treatment monitoring. To improve the monitoring of the AK burden in clinical settings with enhanced automation and precision, the present study evaluates the application of semantic segmentation based on the U-Net architecture (i.e., AKU-Net). AKU-Net employs transfer learning to compensate for the relatively small dataset of annotated images and integrates a recurrent process based on convLSTM to exploit contextual information and address the challenges related to the low contrast and ambiguous boundaries of AK-affected skin regions. We used an annotated dataset of 569 clinical photographs from 115 patients with actinic keratosis to train and evaluate the model. From each photograph, patches of 512 × 512 pixels were extracted using translation lesion boxes that encompassed lesions in different positions and captured different contexts of perilesional skin. In total, 16,488 translation-augmented crops were used for training the model, and 403 lesion center crops were used for testing. To demonstrate the improvements in AK detection, AKU-Net was compared with plain U-Net and U-Net++ architectures. The experimental results highlighted the effectiveness of AKU-Net, improving upon both automation and precision over existing approaches, paving the way for more effective and reliable evaluation of actinic keratosis in clinical settings.

Keywords: U-Net; actinic keratosis; clinical photography; cutaneous cancerization field; deep learning; semantic segmentation; skin lesions.

Figure 1

U-Net architecture proposed by Ronneberger et al. (2015) [38].

Figure 2

VGG16 backbone transfer learning scheme for the U-Net encoder.

Figure 3

Proposed model for AK detection based on U-Net architecture (AKU-Net), with three key modifications: a utilizing the pretrained VGG16 as the encoder, incorporating convLSTM processing units in the skip connections, and integrating the BN in the decoding layer.

Figure 4

A patch from a clinical photograph with AK (left) and the predicted area (right). The AK labeling from the system is highlighted in red, and the yellow line represents the expert’s annotation. The estimations for the $Dice$, $IOU$, and $aF1$ coefficients were 0.76, 0.62, and 0.97, respectively.

Figure 5

Photographs were taken to capture the presence of multiple lesions across the entire face and provide different views of the same lesions. With green color are the annotated by the experts AK lesions. The white circular sticker is a fiducial marker with a diameter of ¼ inch.

Figure 6

From left to right: 512 × 512 lesion center crop and the corresponding translation-augmented lesion crops.

Figure 7

A visual demonstration of the efficiency of the three trained models in AK detection in challenging skin areas. Skin folds, hairs, and small vessels all constituted sources of severe false positives for AKCNN (a–d). AK lesions with a low contrast and ambiguous boundaries (e,f) were successfully detected by AKU-Net. Note the inclusion of lesions from almost the whole spectrum of clinical AK grades. The experts’ annotations are in yellow, and the models’ predictions in red.

Figure 8

Exemplary visualization of AK detection of the same frame (Table 3; Frame 3) with two model architectures. (Left) The performance of AKCNN, where the “scanning” area (black line) was manually predefined to exclude areas covered by hairs and the anatomical structure of eyes: blue lines are the expert-annotated AK lesions and scanty colored areas correspond to the detected AK. (Right) Detection of AK using AKU-Net in the entire frame region (blue box). Note the aggregation of the AK-affected skin area in four distinct patches (red color).

Figure 9

Exemplary visualization of AK detection of the same frame (Table 3; Frame 6) with two model architectures. (Left) AKCNN detection results with the highest false-positive rate$aPrec=0.26$. (Right) The AKU-Net was favorably tolerant of the selection of the scanning area that was simply either a boxed area (blue box; $aPrec=0.71)$ or considered as a wider frame ($aPrec=0.69$).