Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Jul 10;22(7):e18697.
doi: 10.2196/18697.

Diagnosing Parkinson Disease Through Facial Expression Recognition: Video Analysis

Bo Jin  1 , Yue Qu  1 , Liang Zhang  2 , Zhan Gao  3
Affiliations

Diagnosing Parkinson Disease Through Facial Expression Recognition: Video Analysis

Bo Jin et al. J Med Internet Res. .

Abstract

Background: The number of patients with neurological diseases is currently increasing annually, which presents tremendous challenges for both patients and doctors. With the advent of advanced information technology, digital medical care is gradually changing the medical ecology. Numerous people are exploring new ways to receive a consultation, track their diseases, and receive rehabilitation training in more convenient and efficient ways. In this paper, we explore the use of facial expression recognition via artificial intelligence to diagnose a typical neurological system disease, Parkinson disease (PD).

Objective: This study proposes methods to diagnose PD through facial expression recognition.

Methods: We collected videos of facial expressions of people with PD and matched controls. We used relative coordinates and positional jitter to extract facial expression features (facial expression amplitude and shaking of small facial muscle groups) from the key points returned by Face++. Algorithms from traditional machine learning and advanced deep learning were utilized to diagnose PD.

Results: The experimental results showed our models can achieve outstanding facial expression recognition ability for PD diagnosis. Applying a long short-term model neural network to the positions of the key features, precision and F1 values of 86% and 75%, respectively, can be reached. Further, utilizing a support vector machine algorithm for the facial expression amplitude features and shaking of the small facial muscle groups, an F1 value of 99% can be achieved.

Conclusions: This study contributes to the digital diagnosis of PD based on facial expression recognition. The disease diagnosis model was validated through our experiment. The results can help doctors understand the real-time dynamics of the disease and even conduct remote diagnosis.

Keywords: Parkinson disease; artificial intelligence; face landmarks; machine learning.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Smiles of a patient with Parkinson disease (left) and person without Parkinson disease (right).
Figure 2
Figure 2
Face key points (n=106) returned by the Face++ interface.
Figure 3
Figure 3
Relative face coordinate system, where the Cartesian, or absolute, coordinate system is represented by the black coordinate system, which was used to record the position of pixels in the image, and the non-Cartesian, or relative, coordinate system is represented by the blue coordinate system, which was used to record the relative position of key points on the face.
Figure 4
Figure 4
Coordinate system conversion, where the absolute coordinates (m1,n1), (m2, n2), (a1, a2), and (b1, b2) can be converted to the relative coordinates (0,0), (x,y), (1,0), and (0,1), respectively.
Figure 5
Figure 5
The effects of least absolute shrinkage and selection operator (LASSO) feature compression on logistic regression (LR) and support vector machine (SVM) models.
Figure 6
Figure 6
The key points that have a large influence on the classification result.
See this image and copyright information in PMC

References

    1. Lexell J. Evidence for nervous system degeneration with advancing age. J Nutr. 1997 May;127(5 Suppl):1011S–1013S. doi: 10.1093/jn/127.5.1011S. doi: 10.1093/jn/127.5.1011S. - DOI - DOI - PubMed
    1. Thomson R. Disease briefing: Parkinson's disease. Journal of International Pharmaceutical Research. 2015;(3):338–345. doi: 10.13220/j.cnki.jipr.2015.03.009. - DOI
    1. Martinez-Martín P, Rodriguez-Blazquez C, Paz S, Forjaz MJ, Frades-Payo B, Cubo E, de Pedro-Cuesta J, Lizán L, ELEP Group Parkinson Symptoms and Health Related Quality of Life as Predictors of Costs: A Longitudinal Observational Study with Linear Mixed Model Analysis. PLoS One. 2015 Sep;10(12):e0145310–33. doi: 10.1371/journal.pone.0145310. http://dx.plos.org/10.1371/journal.pone.0145310 - DOI - DOI - PMC - PubMed
    1. Kruszka P, Addissie YA, McGinn DE, Porras AR, Biggs E, Share M, Crowley TB, Chung BHY, Mok GTK, Mak CCY, Muthukumarasamy P, Thong MK, Sirisena ND, Dissanayake VHW, Paththinige CS, Prabodha LBL, Mishra R, Shotelersuk V, Ekure EN, Sokunbi OJ, Kalu N, Ferreira CR, Duncan JM, Patil SJ, Jones KL, Kaplan JD, Abdul-Rahman OA, Uwineza A, Mutesa L, Moresco A, Obregon MG, Richieri-Costa A, Gil-da-Silva-Lopes VL, Adeyemo AA, Summar M, Zackai EH, McDonald-McGinn DM, Linguraru MG, Muenke M. 22q11.2 deletion syndrome in diverse populations. Am J Med Genet A. 2017 Apr;173(4):879–888. doi: 10.1002/ajmg.a.38199. http://europepmc.org/abstract/MED/28328118 - DOI - PMC - PubMed
    1. Basel-Vanagaite L, Wolf L, Orin M, Larizza L, Gervasini C, Krantz ID, Deardoff MA. Recognition of the Cornelia de Lange syndrome phenotype with facial dysmorphology novel analysis. Clin Genet. 2016 May;89(5):557–63. doi: 10.1111/cge.12716. doi: 10.1111/cge.12716. - DOI - DOI - PubMed

Publication types