Clinical Investigation of a Rapid Non-invasive Multispectral Imaging Device Utilizing an Artificial Intelligence Algorithm for Improved Burn Assessment

Abstract

Currently, the incorrect judgment of burn depth remains common even among experienced surgeons. Contributing to this problem are change in burn appearance throughout the first week requiring periodic evaluation until a confident diagnosis can be made. To overcome these issues, we investigated the feasibility of an artificial intelligence algorithm trained with multispectral images of burn injuries to predict burn depth rapidly and accurately, including burns of indeterminate depth. In a feasibility study, 406 multispectral images of burns were collected within 72 hours of injury and then serially for up to 7 days. Simultaneously, the subject's clinician indicated whether the burn was of indeterminate depth. The final depth of burned regions within images were agreed upon by a panel of burn practitioners using biopsies and 21-day healing assessments as reference standards. We compared three convolutional neural network architectures and an ensemble in their capability to automatically highlight areas of nonhealing burn regions within images. The top algorithm was the ensemble with 81% sensitivity, 100% specificity, and 97% positive predictive value (PPV). Its sensitivity and PPV were found to increase in a sigmoid shape during the first week postburn, with the inflection point at day 2.5. Additionally, when burns were labeled as indeterminate, the algorithm's sensitivity, specificity, PPV, and negative predictive value were: 70%, 100%, 97%, and 100%. These results suggest multispectral imaging combined with artificial intelligence is feasible for detecting nonhealing burn tissue and could play an important role in aiding the earlier diagnosis of indeterminate burns.

Figure 1.

The multispectral imaging device utilized for this study. The system include: (1) a touch-screen display; (2) mobile cart containing the processing unit; (3) an articulating arm to position the camera; and (4) the MSI subsystem. MSI, multispectral imaging.

Figure 2.

Decision tree used by pathologists to determine burn depth. Superficial burns are unlikely to be sent for excision and biopsy and not included in this decision tree.

Figure 3.

Imaging and ground truth masks from a heterogeneous burn on the dorsal aspect of a subject. Green guiding beams indicate the location and distance of the MSI image; color image of the study burn generated from the MSI data; detailed ground truth provided by expert truthing panel; binary ground truth where all nonhealing burn have been labeled as the target pixels in white. MSI, multispectral imaging.

Figure 4.

CNN results from three subjects. Columns represent: (a) the reference photo; (b) map of severe burn generated by the AI algorithm with color bar indicating probabilities between 0.0 and 1.0 (probabilities < .05 not shown in the image); and (c) the segmented images resulting from the application of a threshold to the probability map. Rows include: (1) a 71-year-old male with a severe flame burn indicated by the highlighted region in column C; (2) a 44-year-old male with a superficial flame burn indicated by a lack of highlighted region in column C; and (3) a 56-year-old male with a severe flame burn. AI, artificial intelligence; CNN, convolutional neural network.

Figure 5.

Results of the Voting Ensemble DL algorithm demonstrating the segmentation of nonhealing burn (ie, deep partial-thickness and full-thickness burn) within the image. (a) Color image of a heterogeneous burn on the back of a study subject. (b) Probability “heat map” of the Voting Ensemble DL algorithm. (c) Predicted area of nonhealing burn after threshold has been applied to the probability heat map in image “b”. (d) Ground truth location of nonhealing burn in the image. (e) Comparison of the ground truth to the DL algorithm indicating the four outcome types for every pixel in the image. DL, deep learning.

Figure 6.

Performance of Voting Ensemble DL algorithm over various thresholds. (Left) PAR curve in which the blue ribbon represents the PI-95 for Recall and the orange ribbon represents the PI-95 for Precision. Area under the PAR curve was 0.81. (Right) receiver operating characteristic curve for comparison looks near perfect with an AUC of 0.99 demonstrating how unbalanced data can bias traditional measures of classifier performance. The larger blue point in each plot indicates the optimum threshold value. DL, deep learning; PAR, precision and recall.

Figure 7.

Performance of the Voting Ensemble DL algorithm over the first week of injury. (Left) This plot shows the change in PPV over time where the black line is the median PPV, and the gray ribbon is the PI-95. Points represent the sensitivity of individual images (Right) demonstrates the increase in Sensitivity over time from about 9 to 99%. Sensitivity shows a sharp increase between 1.5 and 3.5 days since injury whereas PPV increases only modestly throughout the 7-day timespan. DL, deep learning; PPV, positive predictive value.

Figure 8.

Repeated measurements from a nonhealing burn in the study: (Left column) color images showing the edges of the burn began as hyperemic at 24 h postburn (pinkish-red) and then went on to develop an eschar that became more yellow as time went on. (Middle column) predicted probability of nonhealing burn by the Voting Ensemble algorithm at each timepoint based on cross-validation. Dark red represents probability of 100% and dark blue probability of 0%. Probability of nonhealing burn increases as time-since-injury increases. (Right column) ground truth images indicating the true location of nonhealing burn in white.

Figure 9.

Performance of DL algorithm over time with burns assessed as indeterminate indicated by, and burns that were diagnosed by the clinician as. Dashed-line: performance of DL algorithm on the subgroup of indeterminate burns. Solid-line: performance of AI on diagnosed burns. Histograms represent the total number of images at each level of DL performance. DL, deep learning.