Telemedicine supported by Augmented Reality: an interactive guide for untrained people in performing an ECG test

Abstract

Background: In many telemedicine applications, the correct use of medical device at the point of need is essential to provide an appropriate service. Some applications may require untrained people to interact with medical devices and patients: care delivery in transportation, military actions, home care and telemedicine training.Appropriate operation of medical device and correct connection with patient's body are crucial. In these scenarios, tailored applications of Augmented Reality can offer a valid support by guiding untrained people at the point of need. This study aims to explore the feasibility of using Augmented Reality in telemedicine applications, by facilitating an acceptable use of biomedical equipment by any unskilled person. In particular, a prototype system was built in order to estimate how untrained users, with limited or no knowledge, can effectively interact with an ECG device and properly placing ECG electrodes on patient's chest.

Methods: An Augmented Reality application was built to support untrained users in performing an ECG test. Simple markers attached to the ECG device and onto patient's thorax allow camera calibration. Once objects and their pose in the space are recognized, the video of the current scene is enriched, in real-time, with additional pointers, text boxes and audio that help the untrained operator to perform the appropriate sequence of operations. All the buttons, switches, ports of the ECG device together with the location of precordial leads were coded and indicated. Some user's voice commands were also included to improve usability.

Results: Ten untrained volunteers, supported by the augmented reality, were able to carry out a complete ECG test first on a mannequin and then on a real patient in a reasonable time (about 8 minutes on average). Average positioning errors of precordial electrodes resulted less than 3 mm for the mannequin and less than 7 mm for the real patient. These preliminary findings suggest the effectiveness of the developed application and the validity of clinical ECG recordings.

Conclusion: This application can be adapted to support the use of other medical equipment as well as other telemedicine tasks and it could be performed with a Tablet or a Smartphone.

Figure 1

System architecture. Scheme of the operations of the developed AR application to guide an untrained user to correctly perform an ECG test.

Figure 2

Portable ECG with markers. The portable ECG device adopted for this study with attached on the correspondent three markers. The side of each marker measures 4 cm (a). The rigid T-shaped frame enclosing the three markers placed on the thorax of a mannequin. The upper side of the lowest marker is placed in correspond to the xiphoid process (b). Schematic representation of the reference frame adopted in the trials and the relative positions of the precordial electrodes V1, V2, V3, V4, V5 and V6 (c).

Figure 3

Untrained user on a mannequin and on the portable ECG. Pictures representing an untrained user while placing precordial electrodes on the mannequin (a) and operating the ECG-device (b).

Figure 4

Examples of augmented scenes. Examples of augmented real scenes: the power button of the ECG device is surrounded by a light green rectangle, while audio and text invite the user to press it (a); a little red spot indicates the LED to be checked by the users (b); the location of the V3 precordial electrode on the mannequin is highlighted with a red circle (c); the location of the V6 precordial electrode is highlighted with a red circle (d) please note the inclination with respect to the plane of the markers; the location of the V3 precordial electrode on a real patient is highlighted with a red circle (e); placement of the V5 precordial electrode on the patient’s thorax, the location of V5 is highlighted with a red circle (f).

Figure 5

Average x, y and z components of positioning errors for precordial electrodes. Averages (bars) and Standard Deviations (segments) of the errors (distance from the expected location reported in centimeters for each of the x, y and z axes of the space) in the placement of the precordial electrodes on the mannequin (a) and on the patient’s chest (b) by the ten untrained users. V1, V2, V3, V4, V5 and V6 are the six precordial locations;

Figure 6

An improved marker set. Picture of the improved T-structure including the two extra markers oriented perpendicularly to the patient’s coronal plane.

Figure 7

Possible AR application on smartphones. Possible, future scenario involving the use of the proposed AR application on smartphones.