Automated facial recognition system using deep learning for pain assessment in adults with cerebral palsy

Abstract

Objective: Assessing pain in individuals with neurological conditions like cerebral palsy is challenging due to limited self-reporting and expression abilities. Current methods lack sensitivity and specificity, underlining the need for a reliable evaluation protocol. An automated facial recognition system could revolutionize pain assessment for such patients.The research focuses on two primary goals: developing a dataset of facial pain expressions for individuals with cerebral palsy and creating a deep learning-based automated system for pain assessment tailored to this group.

Methods: The study trained ten neural networks using three pain image databases and a newly curated CP-PAIN Dataset of 109 images from cerebral palsy patients, classified by experts using the Facial Action Coding System.

Results: The InceptionV3 model demonstrated promising results, achieving 62.67% accuracy and a 61.12% F1 score on the CP-PAIN dataset. Explainable AI techniques confirmed the consistency of crucial features for pain identification across models.

Conclusion: The study underscores the potential of deep learning in developing reliable pain detection systems using facial recognition for individuals with communication impairments due to neurological conditions. A more extensive and diverse dataset could further enhance the models' sensitivity to subtle pain expressions in cerebral palsy patients and possibly extend to other complex neurological disorders. This research marks a significant step toward more empathetic and accurate pain management for vulnerable populations.

Keywords: Pain assessment; automated facial recognition; cerebral palsy; deep learning; pain expression image dataset.

Figure 1.

Example of images with and without pain, from the four datasets employed in this study.

Figure 2.

Resulting images after the pre-processing performed before any training or testing on the four databases, featuring a face crop and background subtraction.

Figure 3.

Visualization of the explanation process followed for the identification of the important regions for a specific model to classify into a class.

Figure 4.

Accuracy (top) and F1 score (bottom) of the pain prediction results of the three different trainings performed, displayed by network. The values correspond to the average between the five validation splits, for each training scenario.

Figure 5.

Precision (top) and recall (bottom) of the pain prediction results by class on the CP-PAIN dataset, displayed by network. The values correspond to the average between the five validation splits, for each training scenario.

Figure 6.

Heatmaps representing face regions importance for the different models, classes, and databases.

Figure 7.

Examples of images correctly classified as “pain” by the InceptionV3 model on the CP-PAIN dataset.

Figure 8.

Examples of images incorrectly classified as “no pain” by the InceptionV3 model on the CP-PAIN dataset.

Figure 9.

Examples of images correctly classified as “pain” by the InceptionV3 model on the PAIN-DB dataset, grouped by original dataset (by rows).

Figure 10.

Examples of images incorrectly classified as “no pain” by the InceptionV3 model on the PAIN-DB dataset, grouped by original dataset (by rows).