Evaluation of the educational effectiveness of a virtual reality intravenous insertion simulator

Abstract

Objective: To evaluate construct and content validity as well as learners' perceptions of CathSim, a virtual reality intravenous (IV) insertion simulator.

Methods: A prospective cohort study design was employed to determine construct validity, and a participant survey was used to ascertain content validity as well as user perceptions of CathSim. Forty-one attendings, residents, and medical students in emergency medicine and anesthesia attempted five simulated IV insertions on CathSim. Subject performances were scored by the computer, and subject perceptions of the simulator were measured using a Likert scale questionnaire (1 = worst rating; 5 = best rating). The subjects were divided into three groups (novices, intermediates, and experts) based on previous IV experience. To determine construct validity, performances of the three groups were compared using one-way analysis of variance (ANOVA). To determine content validity, the experts' perceptions of the simulator's realism and usefulness were assessed. Study subjects' perceptions of the simulator's ease of use and overall appeal were analyzed.

Results: The experts scored better than the others in five of nine scoring parameters (p < 0.05). The experts rated the realism of CathSim's four major simulation components at 3.85, 3.46, 3.69, and 3.46; the overall realism of CathSim at 2.93; and its utility for medical student training at 4.57. The simulator's ease of use was rated at 2.34 by all subjects. Novices reported a score of 4.59 regarding their likelihood to use the simulator.

Conclusions: CathSim demonstrated construct validity in five of nine internal scoring parameters and was judged to be adequately realistic and highly useful for medical student training. Despite being difficult to learn to use, it remained appealing to the users, especially the novices.