Evaluation of a Mobile Telesimulation Unit to Train Rural and Remote Practitioners on High-Acuity Low-Occurrence Procedures: Pilot Randomized Controlled Trial

Abstract

Background: The provision of acute medical care in rural and remote areas presents unique challenges for practitioners. Therefore, a tailored approach to training providers would prove beneficial. Although simulation-based medical education (SBME) has been shown to be effective, access to such training can be difficult and costly in rural and remote areas.

Objective: The aim of this study was to evaluate the educational efficacy of simulation-based training of an acute care procedure delivered remotely, using a portable, self-contained unit outfitted with off-the-shelf and low-cost telecommunications equipment (mobile telesimulation unit, MTU), versus the traditional face-to-face approach. A conceptual framework based on a combination of Kirkpatrick's Learning Evaluation Model and Miller's Clinical Assessment Framework was used.

Methods: A written procedural skills test was used to assess Miller's learning level- knows-at 3 points in time: preinstruction, immediately postinstruction, and 1 week later. To assess procedural performance (shows how), participants were video recorded performing chest tube insertion before and after hands-on supervised training. A modified Objective Structured Assessment of Technical Skills (OSATS) checklist and a Global Rating Scale (GRS) of operative performance were used by a blinded rater to assess participants' performance. Kirkpatrick's reaction was measured through subject completion of a survey on satisfaction with the learning experiences and an evaluation of training.

Results: A total of 69 medical students participated in the study. Students were randomly assigned to 1 of the following 3 groups: comparison (25/69, 36%), intervention (23/69, 33%), or control (21/69, 31%). For knows, as expected, no significant differences were found between the groups on written knowledge (posttest, P=.13). For shows how, no significant differences were found between the comparison and intervention groups on the procedural skills learning outcomes immediately after the training (OSATS checklist and GRS, P=1.00). However, significant differences were found for the control versus comparison groups (OSATS checklist, P<.001; GRS, P=.02) and the control versus intervention groups (OSATS checklist, P<.001; GRS, P=.01) on the pre- and postprocedural performance. For reaction, there were no statistically significant differences between the intervention and comparison groups on the satisfaction with learning items (P=.65 and P=.79) or the evaluation of the training (P=.79, P=.45, and P=.31).

Conclusions: Our results demonstrate that simulation-based training delivered remotely, applying our MTU concept, can be an effective way to teach procedural skills. Participants trained remotely in the MTU had comparable learning outcomes (shows how) to those trained face-to-face. Both groups received statistically significant higher procedural performance scores than those in the control group. Participants in both instruction groups were equally satisfied with their learning and training (reaction). We believe that mobile telesimulation could be an effective way of providing expert mentorship and overcoming a number of barriers to delivering SBME in rural and remote locations.

Keywords: assessment; chest tubes; distributed medical education; emergency medicine; medical education; remote-facilitation; rural health; simulation training.

Figure 1

Framework for Learning Assessment, based on Kirkpatrick (left) and Miller (right). Adapted from Dubrowski et al [26].

Figure 2

The mobile telesimulation unit rapid deployment tent.

Figure 3

Overview of the setup for the mentor and the trainees in the mobile telesimulation unit.

Figure 4

The interior of the mobile telesimulation unit demonstrating setup for procedural training.

Figure 5

Study design.

Figure 6

Setup used in the video recording of the chest tube procedure (A) and example of a completed chest tube insertion (B).

Figure 7

Global Rating Scale of operative performance.

Figure 8

Box plots of the modified Objective Structured Assessment of Technical Skills checklist and GRS scores. GRS: Global Rating Scale.