Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Jun 28;21(6):e13594.
doi: 10.2196/13594.

Feasibility of a Real-Time Clinical Augmented Reality and Artificial Intelligence Framework for Pain Detection and Localization From the Brain

Xiao-Su Hu  1   2 , Thiago D Nascimento  1 , Mary C Bender  1 , Theodore Hall  3 , Sean Petty  3 , Stephanie O'Malley  3 , Roger P Ellwood  4 , Niko Kaciroti  1   2   5 , Eric Maslowski  6 , Alexandre F DaSilva  1   2
Affiliations

Feasibility of a Real-Time Clinical Augmented Reality and Artificial Intelligence Framework for Pain Detection and Localization From the Brain

Xiao-Su Hu et al. J Med Internet Res. .

Abstract

Background: For many years, clinicians have been seeking for objective pain assessment solutions via neuroimaging techniques, focusing on the brain to detect human pain. Unfortunately, most of those techniques are not applicable in the clinical environment or lack accuracy.

Objective: This study aimed to test the feasibility of a mobile neuroimaging-based clinical augmented reality (AR) and artificial intelligence (AI) framework, CLARAi, for objective pain detection and also localization direct from the patient's brain in real time.

Methods: Clinical dental pain was triggered in 21 patients by hypersensitive tooth stimulation with 20 consecutive descending cold stimulations (32°C-0°C). We used a portable optical neuroimaging technology, functional near-infrared spectroscopy, to gauge their cortical activity during evoked acute clinical pain. The data were decoded using a neural network (NN)-based AI algorithm to classify hemodynamic response data into pain and no-pain brain states in real time. We tested the performance of several networks (NN with 7 layers, 6 layers, 5 layers, 3 layers, recurrent NN, and long short-term memory network) upon reorganized data features on pain diction and localization in a simulated real-time environment. In addition, we also tested the feasibility of transmitting the neuroimaging data to an AR device, HoloLens, in the same simulated environment, allowing visualization of the ongoing cortical activity on a 3-dimensional brain template virtually plotted on the patients' head during clinical consult.

Results: The artificial neutral network (3-layer NN) achieved an optimal classification accuracy at 80.37% (126,000/156,680) for pain and no pain discrimination, with positive likelihood ratio (PLR) at 2.35. We further explored a 3-class localization task of left/right side pain and no-pain states, and convolutional NN-6 (6-layer NN) achieved highest classification accuracy at 74.23% (1040/1401) with PLR at 2.02.

Conclusions: Additional studies are needed to optimize and validate our prototype CLARAi framework for other pains and neurologic disorders. However, we presented an innovative and feasible neuroimaging-based AR/AI concept that can potentially transform the human brain into an objective target to visualize and precisely measure and localize pain in real time where it is most needed: in the doctor's office.

International registered report identifier (irrid): RR1-10.2196/13594.

Keywords: artificial intelligence; pain; spectroscopy, near-infrared; virtual reality.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: The content described within this study has been developed at the UM and disclosed to the UM Office of Technology Transfer. All intellectual property rights including but not limited to patents/patent applications, trademark and copyright of software, algorithms, reports, displays, and visualizations are owned by the Regents of the University of Michigan. Drs DaSilva and Maslowski are the co-creators of CLARAi. Drs DaSilva and Malowski are co-founders of MoxyTech Inc, which has optioned the technology CLARAi from University of Michigan. Dr. Roger P Ellwood, who was previously an employee of the Colgate Palmolive company-the funding agency of this work.

Figures

Figure 1
Figure 1
Experiment flow chart. The green line indicates the convolutional neural network with 7 layers (CNN-7), the blue line indicates the CNN network with 6 layers, the orange line indicates the CNN network with 5 layers (CNN-5), the red line indicates the long short-term memory network, the dark blue line indicates the recurrent NN, and the yellow line indicates the artificial NN with 3 layers for experiment 1—pain/no-pain prediction and experiment 2—left/right pain localization task. Experiment 1 included the data collected from N=12 participants, 239 trials in total, whereas experiment 2 included the data collected from N=2 participants, 20 trials in total. CNN: convolutional neural network.
Figure 2
Figure 2
Study framework.
Figure 3
Figure 3
Representative averaged oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) heat map from all data channels. The upper and lower panels, respectively, indicated the hemodynamic responses during pain and no-pain statues. The left and right panels, respectively, indicated the HbO and HbR responses. The red and blue circles, respectively, highlighted 2 regions of interest, sensory and prefrontal cortices. HbO: oxygenated hemoglobin; HbR: deoxygenated hemoglobin; PFC: prefrontal cortex.
Figure 4
Figure 4
CLARAi framework that integrated clinical real-time neuroimaging, augmented reality, and artificial intelligence provides an augmented clinical environment by displaying neuroimaging data with predicted and localized pain of patient. The classification codes for no-pain, right side pain, and left side pain was defined as 0, 1, and 2, respectively, for model training purposes.
See this image and copyright information in PMC

References

    1. Gaskin DJ, Richard P. The economic costs of pain in the United States. J Pain. 2012 Aug;13(8):715–24. doi: 10.1016/j.jpain.2012.03.009.S1526-5900(12)00559-7 - DOI - PubMed
    1. Nascimento TD, DosSantos MF, Danciu T, DeBoer M, van Holsbeeck H, Lucas SR, Aiello C, Khatib L, Bender MA, UMSoD (Under)Graduate Class Of 2014. Zubieta J, DaSilva AF. Real-time sharing and expression of migraine headache suffering on Twitter: a cross-sectional infodemiology study. J Med Internet Res. 2014 Apr;16(4):e96. doi: 10.2196/jmir.3265. http://www.jmir.org/2014/4/e96/ v16i4e96 - DOI - PMC - PubMed
    1. Wager TD, Atlas LY, Lindquist MA, Roy M, Woo C, Kross E. An fMRI-based neurologic signature of physical pain. N Engl J Med. 2013 Apr 11;368(15):1388–97. doi: 10.1056/NEJMoa1204471. http://europepmc.org/abstract/MED/23574118 - DOI - PMC - PubMed
    1. Harper DE, Shah Y, Ichesco E, Gerstner GE, Peltier SJ. Multivariate classification of pain-evoked brain activity in temporomandibular disorder. Pain Rep. 2016 Sep;1(3) doi: 10.1097/PR9.0000000000000572. http://europepmc.org/abstract/MED/28630949 - DOI - PMC - PubMed
    1. Davis KD, Flor H, Greely HT, Iannetti GD, Mackey S, Ploner M, Pustilnik A, Tracey I, Treede R, Wager TD. Brain imaging tests for chronic pain: medical, legal and ethical issues and recommendations. Nat Rev Neurol. 2017 Oct;13(10):624–638. doi: 10.1038/nrneurol.2017.122.nrneurol.2017.122 - DOI - PubMed

Publication types