Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Apr 29:5:815333.
doi: 10.3389/frai.2022.815333. eCollection 2022.

Using Facial Landmark Detection on Thermal Images as a Novel Prognostic Tool for Emergency Departments

Ruben Baskaran  1 , Karim Møller  1 , Uffe Kock Wiil  1 , Mikkel Brabrand  2
Affiliations

Using Facial Landmark Detection on Thermal Images as a Novel Prognostic Tool for Emergency Departments

Ruben Baskaran et al. Front Artif Intell. .

Abstract

Introduction: Emergency departments (ED) at hospitals sometimes experience unexpected deterioration in patients that were assessed to be in a stable condition upon arrival. Odense University Hospital (OUH) has conducted a retrospective study to investigate the possibilities of prognostic tools that can detect these unexpected deterioration cases at an earlier stage. The study suggests that the temperature difference (gradient) between the core and the peripheral body parts can be used to detect these cases. The temperature between the patient's inner canthus (core temperature) and the tip of the nose (peripheral temperature) can be measured with a thermal camera. Based on the temperature measurement from a thermal image, a gradient value can be calculated, which can be used as an early indicator of potential deterioration.

Problem: The lack of a tool to automatically calculate the gradient has prevented the ED at OUH in conducting a comprehensive prospective study on early indicators of patients at risk of deterioration. The current manual way of doing facial landmark detection on thermal images is too time consuming and not feasible as part of the daily workflow at the ED, where nurses have to triage patients within a few minutes.

Objective: The objective of this study was to automate the process of calculating the gradient by developing a handheld prognostic tool that can be used by nurses for automatically performing facial landmark detection on thermal images of patients as they arrive at the ED.

Methods: A systematic literature review has been conducted to investigate previous studies that have been done for applying computer vision methods on thermal images. Several meetings, interviews and field studies have been conducted with the ED at OUH in order to understand their workflow, formulate and prioritize requirements and co-design the prognostic tool.

Results: The study resulted in a novel Android app that can capture a thermal image of a patient's face with a thermal camera attached to a smartphone. Within a few seconds, the app then automatically calculates the gradient to be used in the triage process. The developed tool is the first of its kind using facial landmark detection on thermal images for calculating a gradient that can serve as a novel prognostic indicator for ED patients.

Keywords: computer vision; machine learning; prognosis; thermal imaging; triage.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
Overview of the parameters that are used to triage the patients (Schmidt and Wiil, 2015).
Figure 2
Figure 2
Overview of the proposed solution. Source: https://fontawesome.com/search?m=free.
Figure 3
Figure 3
Overview of setup for systematic literature review.
Figure 4
Figure 4
Overview of filtering process and results from systematic literature review.
Figure 5
Figure 5
Idea behind the CNN algorithms.
Figure 6
Figure 6
Idea behind the RGB-thermal mapping algorithm.
Figure 7
Figure 7
Idea behind the max-min template algorithm.
Figure 8
Figure 8
Flow chart overview of the android application.
Figure 9
Figure 9
Overview of all the views in the android application.
Figure 10
Figure 10
Output markers for the CNN algorithm.
Figure 11
Figure 11
Output markers for the CNN with transfer learning algorithm.
Figure 12
Figure 12
Output markers for the RGB-Thermal Mapping algorithm.
Figure 13
Figure 13
Output markers for the Max-Min Template algorithm.
See this image and copyright information in PMC

References

    1. Alkali A. H., Saatchi R., Elphick H., Burke D. (2014). Eyes' corners detection in infrared images for real-time noncontact respiration rate monitoring, in 2014 World Congress on Computer Applications and Information Systems (WCCAIS), Hammamet, 1–5.
    1. Al-Khalidi F. Q., Saatchi R., Burke D., Elphick H. (2010). Tracking human face features in thermal images for respiration monitoring, ACS/IEEE International Conference on Computer Systems and Applications - AICCSA 2010, Hammamet, 1–6.
    1. Ashrant Aryal, Burcin Becerik-Gerber. (2019). Skin temperature extraction using facial landmark detection and thermal imaging for comfort assessment, in Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys '19). New York, NY: Association for Computing Machinery, 71–80.
    1. Brabrand M., Nissen S. K., Hanson S., Fløjstrup M. (2021). Clinical thermography at extreme temperatures. Acute Med. 20, 236. 10.52964/AMJA.0872 - DOI - PubMed
    1. Budzan S., Wyzgolik R. (2013). Face and eyes localization algorithm in thermal images for temperature measurement of the inner canthus of the eyes. Infr. Phys. Technol. 60, 225–234. 10.1016/j.infrared.2013.05.007 - DOI - PMC - PubMed

LinkOut - more resources