Evaluation Challenges for the Application of Extended Reality Devices in Medicine

Ryan Beams¹, Ellenor Brown², Wei-Chung Cheng², Janell S Joyner², Andrea S Kim², Kimberly Kontson², Dimitri Amiras³, Tassilo Baeuerle⁴, Walter Greenleaf⁵, Rafael J Grossmann⁶, Atul Gupta⁷, Christoffer Hamilton⁸, Hong Hua⁹, Tran Tu Huynh¹⁰, Christoph Leuze¹¹, Sarah B Murthi¹², John Penczek^{13

14}, Jennifer Silva^{15

16}, Brennan Spiegel¹⁷, Amitabh Varshney¹⁸, Aldo Badano²

Affiliations

¹ Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA. Ryan.Beams@fda.hhs.gov.
² Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA.
³ Department of Imaging, Imperial College Healthcare NHS Trust, London, UK.
⁴ CognifiSense, Inc., Sunnyvale, CA, USA.
⁵ Stanford University Virtual Human Interaction Lab, Stanford University, Stanford, CA, USA.
⁶ Northern Light Health, Brewer, ME, USA.
⁷ Philips, Cambridge, MA, USA.
⁸ Brainlab AG, Munich, Germany.
⁹ James C. Wyant College of Optical Sciences, University of Arizona, Tucson, AZ, USA.
¹⁰ OpticSurg Inc., Wilmington, DE, USA.
¹¹ Department of Radiology, Stanford University, Stanford, CA, USA.
¹² R Adams Cowley Shock Trauma Center, University of Maryland Baltimore, Baltimore, MD, USA.
¹³ NIST, Boulder, CO, USA.
¹⁴ University of Colorado, Boulder, CO, USA.
¹⁵ SentiAR, Inc., St Louis, MT, USA.
¹⁶ School of Medicine, Division of Pediatric Cardiology, Washington University, St Louis, MO, USA.
¹⁷ Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
¹⁸ Department of Computer Science, University of Maryland, College Park, MD, USA.

PMID: 35469355
PMCID: PMC9582055
DOI: 10.1007/s10278-022-00622-x

Evaluation Challenges for the Application of Extended Reality Devices in Medicine

Ryan Beams et al. J Digit Imaging. 2022 Oct.

. 2022 Oct;35(5):1409-1418.

doi: 10.1007/s10278-022-00622-x. Epub 2022 Apr 25.

Authors

Affiliations

¹ Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA. Ryan.Beams@fda.hhs.gov.
² Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, MD, USA.
³ Department of Imaging, Imperial College Healthcare NHS Trust, London, UK.
⁴ CognifiSense, Inc., Sunnyvale, CA, USA.
⁵ Stanford University Virtual Human Interaction Lab, Stanford University, Stanford, CA, USA.
⁶ Northern Light Health, Brewer, ME, USA.
⁷ Philips, Cambridge, MA, USA.
⁸ Brainlab AG, Munich, Germany.
⁹ James C. Wyant College of Optical Sciences, University of Arizona, Tucson, AZ, USA.
¹⁰ OpticSurg Inc., Wilmington, DE, USA.
¹¹ Department of Radiology, Stanford University, Stanford, CA, USA.
¹² R Adams Cowley Shock Trauma Center, University of Maryland Baltimore, Baltimore, MD, USA.
¹³ NIST, Boulder, CO, USA.
¹⁴ University of Colorado, Boulder, CO, USA.
¹⁵ SentiAR, Inc., St Louis, MT, USA.
¹⁶ School of Medicine, Division of Pediatric Cardiology, Washington University, St Louis, MO, USA.
¹⁷ Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
¹⁸ Department of Computer Science, University of Maryland, College Park, MD, USA.

PMID: 35469355
PMCID: PMC9582055
DOI: 10.1007/s10278-022-00622-x

Abstract

Augmented and virtual reality devices are being actively investigated and implemented for a wide range of medical uses. However, significant gaps in the evaluation of these medical devices and applications hinder their regulatory evaluation. Addressing these gaps is critical to demonstrating the devices' safety and effectiveness. We outline the key technical and clinical evaluation challenges discussed during the US Food and Drug Administration's public workshop, "Medical Extended Reality: Toward Best Evaluation Practices for Virtual and Augmented Reality in Medicine" and future directions for evaluation method development. Evaluation challenges were categorized into several key technical and clinical areas. Finally, we highlight current efforts in the standards communities and illustrate connections between the evaluation challenges and the intended uses of the medical extended reality (MXR) devices. Participants concluded that additional research is needed to assess the safety and effectiveness of MXR devices across the use cases.

Keywords: Augmented reality; Image quality; Medical imaging; Virtual reality.

PubMed Disclaimer

Conflict of interest statement

Dimitri Amiras: Clinical advisor for MedicaliSight.

Figures

**Fig. 1**
The medical extended reality technology landscape encompasses a wide range of components and connectivity paths between the user, the device hardware and the use environment

**Fig. 2**
a. Input image b. Image on VR HMD c. Image with incorrect IPD d. Image on AR HMD with ambient light. e–h. Magnified images of the red box in a–d

See this image and copyright information in PMC

References

1. Gupta, A., Ruijters, D., Flexman, M.L.: Augmented reality for interventional procedures. In: Digital Surgery, pp. 233–246. Springer (2020)
1. Jud L, Fotouhi J, Andronic O, Aichmair A, Osgood G, Navab N, Farshad M. Applicability of augmented reality in orthopedic surgery-a systematic review. BMC Musculoskeletal Disorders. 2020;21(1):1–13. doi: 10.1186/s12891-020-3110-2. - DOI - PMC - PubMed
1. Southworth MK, Silva JNA, Blume WM, Van Hare GF, Dalal AS, Silva JR. Performance evaluation of mixed reality display for guidance during transcatheter cardiac mapping and ablation. IEEE Journal of Translational Engineering in Health and Medicine. 2020;8:1–10. doi: 10.1109/JTEHM.2020.3007031. - DOI - PMC - PubMed
1. Huang EY, Knight S, Guetter CR, Davis CH, Moller M, Slama E, Crandall M. Telemedicine and telementoring in the surgical specialties: a narrative review. The American Journal of Surgery. 2019;218(4):760–766. doi: 10.1016/j.amjsurg.2019.07.018. - DOI - PubMed
1. Cherkin DC, Sherman KJ, Balderson BH, Cook AJ, Anderson ML, Hawkes RJ, Hansen KE, Turner JA. Effect of mindfulness-based stress reduction vs cognitive behavioral therapy or usual care on back pain and functional limitations in adults with chronic low back pain: a randomized clinical trial. JAMA. 2016;315(12):1240–1249. doi: 10.1001/jama.2016.2323. - DOI - PMC - PubMed

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Evaluation Challenges for the Application of Extended Reality Devices in Medicine

Affiliations

Evaluation Challenges for the Application of Extended Reality Devices in Medicine

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

LinkOut - more resources

Full Text Sources