Evaluation of a web-based asynchronous pediatric emergency medicine learning tool for residents and medical students

Abstract

Objectives: To examine the effectiveness of an asynchronous learning tool consisting of web-based lectures for trainees covering major topics pertinent to pediatric emergency medicine (PEM) and to assess resident and student evaluation of this mode of education.

Methods: PEM faculty and fellows created a 21-lecture, web-based curriculum. These 20-minute online lectures used Microsoft PowerPoint with the voice-over feature. A 75-question test was created to assess the effectiveness of the web-based learning model, administered online before and after the rotation in the pediatric emergency department (PED). All fourth-year medical students and residents (across all specialties) rotating through the PED were required to complete 10 of the 21 lectures during their 1-month rotation. The main outcome variable was difference in score between pre- and post-rotation tests of participants who viewed no lectures and those who viewed at least one lecture. Evaluation of the program was assessed by anonymous survey using 5-point discrete visual analog scales. Responses of 4 or 5 were considered positive for analysis.

Results: One hundred eleven residents and fourth-year medical students participated in the program. An initial 32 completed testing before implementation of the on-line lectures (March 2007-August 2007), and another five did not complete the on-line lectures after implementation (September 2007-February 2008). Seventy-one completed testing and on-line lectures, and all but three completed at least 10 on-line lectures during their rotation. Fourteen of 111 trainees did not complete the pre- or post-test (including two who viewed the lectures). The mean change in score was a 1% improvement from pre-test to post-test for trainees who viewed no lectures and a 6.2% improvement for those who viewed the lectures (mean difference = 5.2%, 95% confidence interval = 2.5% to 7.9%). In the linear regression model, the estimate of the coefficient was 0.43 (p < 0.001), meaning that, for each lecture viewed, post-test score rose by 0.43%. Sixty-nine of 75 test items (92%) had a point biserial correlation greater than 0.15. Thirty of the 72 trainees who completed the online lectures and testing (42%) returned surveys. All were comfortable using the Internet, and 87% (26/30) found the web-site easy to use. All felt that their educational goals were met, and 100% felt that the format would be useful in other areas of education.

Conclusions: Although not a replacement for traditional bedside teaching, the use of web-based lectures as an asynchronous learning tool has a positive effect on medical knowledge test scores. Trainees were able to view online lectures on their own schedules, in the location of their choice. This is helpful in a field with shift work, in which trainees rarely work together, making it difficult to synchronously provide lectures to all trainees.