Automated Extraction of Skin Wound Healing Biomarkers From In Vivo Label-Free Multiphoton Microscopy Using Convolutional Neural Networks

Abstract

Background and objectives: Histological analysis is a gold standard technique for studying impaired skin wound healing. Label-free multiphoton microscopy (MPM) can provide natural image contrast similar to histological sections and quantitative metabolic information using NADH and FAD autofluorescence. However, MPM analysis requires time-intensive manual segmentation of specific wound tissue regions limiting the practicality and usage of the technology for monitoring wounds. The goal of this study was to train a series of convolutional neural networks (CNNs) to segment MPM images of skin wounds to automate image processing and quantification of wound geometry and metabolism.

Study design/materials and methods: Two CNNs with a 4-layer U-Net architecture were trained to segment unstained skin wound tissue sections and in vivo z-stacks of the wound edge. The wound section CNN used 380 distinct MPM images while the in vivo CNN used 5,848 with both image sets being randomly distributed to training, validation, and test sets following a 70%, 20%, and 10% split. The accuracy of each network was evaluated on the test set of images, and the effectiveness of automated measurement of wound geometry and optical redox ratio were compared with hand traced outputs of six unstained wound sections and 69 wound edge z-stacks from eight mice.

Results: The MPM wound section CNN had an overall accuracy of 92.83%. Measurements of epidermal/dermal thickness, wound depth, wound width, and % re-epithelialization were within 10% error when evaluated on six full wound sections from days 3, 5, and 10 post-wounding that were not included in the training set. The in vivo wound z-stack CNN had an overall accuracy of 89.66% and was able to isolate the wound edge epithelium in z-stacks from eight mice across post-wound time points to quantify the optical redox ratio within 5% of what was recorded by manual segmentations.

Conclusion: The CNNs trained and presented in this study can accurately segment MPM imaged wound sections and in vivo z-stacks to enable automated and rapid calculation of wound geometry and metabolism. Although MPM is a noninvasive imaging modality well suited to imaging living wound tissue, its use has been limited by time-intensive user segmentation. The use of CNNs for automated image segmentation demonstrate that it is possible for MPM to deliver near real-time quantitative readouts of tissue structure and function. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Keywords: convolutional neural network; deep learning; in vivo; multiphoton microscopy; optical redox ratio; segmentation; wound healing.

Figure 1.

Training diagram for two CNNs using U-Net Architecture to segment MPM imaged wound tissue. The first CNN (A) was trained to segment unstained skin wound cross-sections using endogenous contrast from autofluorescence captured at 755nm ex. / 460±20nm em. (green), 855nm ex. / 525±25nm em. (blue), as well as collagen SHG (red) collected at 855nm. Unique 512×512 pixel MPM images were isolated from 16 tissue sections of murine skin wounds and used for training (n=266), validation (n=76), and an independent test set (n=38). The new CNN was initialized using a transfer learning approach that applied the weights and filters from a previously established network trained to segment H&E stained skin wound sections. Output segmentation classes include the epidermis, dermis, granulation, scab, hair follicles, skeletal muscle, and background. Once trained, the wound section CNN was used to initialize a second network to segment similar tissue regions from in vivo MPM z-stacks taken en face at the wound edge (B). For in vivo network training, a total of 5848 unique 512×512 pixel in vivo MPM images were isolated from 146 z-stacks with 4094 going into a training set, 1170 into a validation set and 584 into a test set. All scale bars shown are 150 µm.

Figure 2.

Accuracy of network segmentation of MPM wound section images. Three representative test set images of skin wound tissue cross-sections from distinct morphological regions are shown with MPM intensity images in the top row, hand traced segmentations in the middle row, and the CNN segmented masks in the bottom row (A). A confusion matrix summarized the pixel-wise classification accuracy of the network for each type of wound tissue based on comparisons to the user traced masks that were used as the ground truth (B). Color-coded classes included granulation tissue (blue), scab (teal), epidermis (green), hair follicles (yellow), muscle (orange), dermis (red), and background (black). Overall network accuracy was 92.83%. Scale bar shown is 150 µm.

Figure 3.

Automated quantification of wound geometry from CNN segmentation of whole tissue sections. To evaluate the accuracy of network segmentation on an entire wound site, 6 wound cross-sections from days 3, 5, and 10 post-injury were imaged with MPM. Intensity images are displayed in the top row, user traced segmentation in the middle row, and the network segmentation in the bottom row (A). A confusion matrix summarized the pixel-wise classification accuracy of the network (88.98% overall) for each type of wound tissue class (B). Geometric wound measurements, such as epidermal/dermal thickness, wound width, wound depth, and percent re-epithelization were automatically calculated from both the user traced and network segmentation results and showed strong agreement (C). Epidermal and dermal thicknesses were also plotted as a function of the long axis of the wound section to compare user traced (dark green/red) and network (light green/red) segmented regions. The Pearson’s correlation coefficient (R) demonstrated strong agreement between the two measurements of thickness for each region. Scale bars shown are 300 µm.

Figure 4.

Accuracy of network segmentation of in vivo wound z-stacks. MPM image z-stacks were acquired en face at the wound edge with successive images in the stack progressing deeper into the wound. Three representative images from different z-stacks in the in vivo test set with varied morphology are shown with MPM intensity images in the top row, user traced segmentation in the middle row, and network segmentation in the bottom row (A). A confusion matrix summarized the pixel-wise classification accuracy of the network for each type of wound tissue class (89.66% overall) based on comparisons to the user traced masks that were used as the ground truth (B). Color-coded classes included granulation tissue (blue), scab (teal), epidermis (green), hair follicles (yellow), dermis (red), and background (black). Scale bar shown is 150 µm.

Figure 5.

CNN results from in vivo MPM z-stacks used to quantify epidermal redox ratio. En face z-stacks at days 3, 5, and 10 post wounding from the same 8 mice (3 stacks/mouse; 24 stacks/day) were segmented by the CNN to isolate the epidermis and quantify an optical redox ratio. Each stack was comprised of 100 individual images (512 × 512 pixels; 584 × 584µm) taken at z-steps of 2.5µm to span a total depth of 250µm.Three images at 25, 50, and 75% of z-stack depth for each time point are shown in the top row with CNN segmented masks in the middle row and the corresponding optical redox ratio maps (jet color map with the epidermis outlined in white) in the bottom row (A). The optical redox ratio using the network segmentation was comparable to values from user traced segmentation (B). The network was capable of segmenting a complete z-stack approximately two orders of magnitude faster than user-guided segmentation (C). Using a 3D Euclidean distance transform to find maximum epidermal thickness from the network segmentation results, the epidermis was found to gradually increased in size significantly between days 3 and 10 (p = 0.0156). All scale bars shown are 150 µm.