Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Feb 13:25:e41992.
doi: 10.2196/41992.

The Clinical Suitability of an Artificial Intelligence-Enabled Pain Assessment Tool for Use in Infants: Feasibility and Usability Evaluation Study

Jeffery David Hughes  1 , Paola Chivers  2   3 , Kreshnik Hoti  4
Affiliations

The Clinical Suitability of an Artificial Intelligence-Enabled Pain Assessment Tool for Use in Infants: Feasibility and Usability Evaluation Study

Jeffery David Hughes et al. J Med Internet Res. .

Abstract

Background: Infants are unable to self-report their pain, which, therefore, often goes underrecognized and undertreated. Adequate assessment of pain, including procedural pain, which has short- and long-term consequences, is critical for its management. The introduction of mobile health-based (mHealth) pain assessment tools could address current challenges and is an area requiring further research.

Objective: The purpose of this study is to evaluate the accuracy and feasibility aspects of PainChek Infant and, therefore, assess its applicability in the intended setting.

Methods: By observing infants just before, during, and after immunization, we evaluated the accuracy and precision at different cutoff scores of PainChek Infant, which is a point-of-care mHealth-based solution that uses artificial intelligence to detect pain and intensity based solely on facial expression. We used receiver operator characteristic analysis to assess interpretability and establish a cutoff score. Clinician comprehensibility was evaluated using a standardized questionnaire. Other feasibility aspects were evaluated based on comparison with currently available observational pain assessment tools for use in infants with procedural pain.

Results: Both PainChek Infant Standard and Adaptive modes demonstrated high accuracy (area under the curve 0.964 and 0.966, respectively). At a cutoff score of ≥2, accuracy and precision were 0.908 and 0.912 for Standard and 0.912 and 0.897 for Adaptive modes, respectively. Currently available data allowed evaluation of 16 of the 17 feasibility aspects, with only the cost of the outcome measurement instrument unable to be evaluated since it is yet to be determined. PainChek Infant performed well across feasibility aspects, including interpretability (cutoff score defined), ease of administration, completion time (3 seconds), and clinician comprehensibility.

Conclusions: This work provides information on the feasibility of using PainChek Infant in clinical practice for procedural pain assessment and monitoring, and demonstrates the accuracy and precision of the tool at the defined cutoff score.

Keywords: PainChek Infant; accuracy; artificial intelligence; assessment; babies; baby; clinical utility; detection; facial; immunization; infant; machine learning; model; newborn; pain; pain assessment; precision; sensitivity; specificity.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: KH and JDH are shareholders in PainChek Ltd (formerly known as EPAT Technologies Ltd), which is commercializing the PainChek Infant. They are also named as coinventors with Mustafa Atee on the patent entitled “A pain assessment method and system,” which has been granted in Australia, China, the United States, and Japan and is pending in Europe and WIPO. KH is employed as a consultant by PainChek Ltd, while also serving as a Professor at the University of Prishtina and University Associate at the Curtin Medical School, Curtin University. JDH is employed as the Chief Scientific Officer of PainChek Ltd and holds an Emeritus Professor appointment at the Curtin Medical School, Curtin University. PC was engaged through DATaR Consulting and was paid as an independent private consultant to undertake the biostatistical analysis for the project by PainChek Ltd. PC also serves as the Deputy Director at the Institute for Health Research, The University of Notre Dame Australia, and holds an Adjunct appointment at the School of Medical and Health Sciences, Edith Cowan University.

Figures

Figure 1
Figure 1
Flow diagram of the study design and setting. NFCS-R: neonatal facial coding system revised; ObsVAS: observer-administered visual analog scale.
Figure 2
Figure 2
Receiver operator characteristic curves of video segments (0=baseline, 1=preparation, 2=vaccine, 3=recovery) for PainChek Adaptive (A) and PainChek Standard (B) modes.
Figure 3
Figure 3
Key steps of the pain assessment process.
Figure 4
Figure 4
Assessor’s evaluations of the feasibility and clinical usability of PainChek Infant, the neonatal facial coding system revised (NFCS-R), and the observer-administered visual analog scale (ObsVAS). *Responses are rated on a 5-point Likert scale, where 0=strongly disagree and 5=strongly agree; results are presented as mean (SD).
Figure 5
Figure 5
PainChek Infant pain monitoring. (A) Profile – recent pain assessments in reverse chronological order, (B) Pain Chart - graphical representation of pain scores versus time.
See this image and copyright information in PMC

References

    1. McMurtry CM. Fact Sheet: procedural pain in children and adolescents. Society of Pediatric Pysychology, Division 54 of the American Society Psychological Association. [2023-01-08]. https://pedpsych.org/fact_sheets/procedural_pain/
    1. Eccleston C, Fisher E, Howard RF, Slater R, Forgeron P, Palermo TM, Birnie KA, Anderson BJ, Chambers CT, Crombez G, Ljungman G, Jordan I, Jordan Z, Roberts C, Schechter N, Sieberg CB, Tibboel D, Walker SM, Wilkinson D, Wood C. Delivering transformative action in paediatric pain: a Lancet Child and Adolescent Health Commission. Lancet Child Adolesc Health. 2021;5(1):47–87. doi: 10.1016/s2352-4642(20)30277-7. - DOI - PubMed
    1. Wood C, von Baeyer CL, Bourrillon A, Dejos-Conant V, Clyti N, Abitbol V. Self-assessment of immediate post-vaccination pain after two different MMR vaccines administered as a second dose in 4- to 6-year-old children. Vaccine. 2004;23(2):127–131. doi: 10.1016/j.vaccine.2004.08.029.S0264-410X(04)00637-1 - DOI - PubMed
    1. Freeman JJ, Spencer AU, Drongowski RA, Vandeven CJM, Apgar B, Teitelbaum DH. Newborn circumcision outcomes: are parents satisfied with the results? Pediatr Surg Int. 2013;30(3):333–338. doi: 10.1007/s00383-013-3430-5. - DOI - PubMed
    1. Taddio A, Katz J, Ilersich AL, Koren G. Effect of neonatal circumcision on pain response during subsequent routine vaccination. Lancet. 1997;349(9052):599–603. doi: 10.1016/s0140-6736(96)10316-0. - DOI - PubMed

Publication types

LinkOut - more resources