Learning Outcomes in a Live Virtual versus In-Person Curriculum for Medical and Pharmacy Students

Abstract

Background: The coronavirus disease (COVID-19) pandemic has been a source of disruption, changing the face of medical education. In response to infection control measures at the University of California, San Diego, the hybrid in-person and recorded preclerkship curriculum was converted to a completely virtual format. The impact of this exclusive virtual teaching platform on the quality of trainee education is unknown.

Objective: To determine the efficacy of a virtual course, relative to traditional hybrid in-person and recorded teaching, and to assess the impact of supplementary educational material on knowledge acquisition.

Methods: A retrospective observational cohort study was performed to assess an introductory course, held mostly in person in 2019 versus completely virtual in 2020, for first-year medical students and second-year pharmacy students at the University of California, San Diego, School of Medicine and Skaggs School of Pharmacy and Pharmaceutical Sciences.

Results: The midterm and final examination scores were similar for the hybrid and virtual courses. There was no association between the hours of recorded lectures watched and final examination scores for either course. In the 2019 in-person and recorded course, students who demonstrated consistent on-time use of practice quizzes scored statistically higher on the final examination (P = 0.0066). In the 2020 virtual course, students who downloaded quizzes regularly had statistically higher scores on the midterm examination (P < 0.0001).

Conclusion: The similar examination scores for the hybrid in-person and recorded and exclusively virtual courses suggest that the short-term knowledge acquired was equivalent, independent of the modality with which the content was delivered. Consistent on-time use of practice quizzes was associated with higher examination scores. Future studies are needed to assess the difference between a completely in-person versus virtual curriculum.

Keywords: coronavirus disease; medical education; preclerkship learning; virtual learning.

Figure 1.

Examination scores between the hybrid in-person and recorded course and coronavirus disease (COVID-19) virtual format courses. (A) Midterm examination scores (P = 0.58). The red line represents the mean. (B) Final examination scores (P = 0.19).

Figure 2.

Relationship between lecture recording watched and examination scores. Students were separated into 6-hour increments for the 2019 hybrid in-person and recorded course and 5-hour increments for the 2020 CVF course. (A) Number of students in each 6-hour block in the 2019 hybrid in-person and recorded course. The orange line represents the total number of lecture hours (36.08 h). (B) Number of students in each 5-hour block in the 2020 CVF course. The orange line represents the total number of lecture hours (30.15 h). (C) Final examination scores of students in each 6-hour block for the 2019 hybrid in-person and recorded course. (D) Final examination scores of students in each 5-hour block for the 2020 CVF course. (E) Students were categorized by the number of lecture recordings viewed in the 2019 hybrid in-person and recorded course. For example, 18 students watched one or two lectures (“2” on the x-axis). (F) Students were categorized by the number of lecture recordings watched in the 2020 CVF course. (G) Percentage of the class that watched each lecture in the 2019 hybrid in-person and recorded course. The 38 lectures are denoted L1–L38. The green line represents the average of the 38 percentages generated across the lectures for the 2019 hybrid in-person and recorded course (40 ± 13%). (H) Percentage of the class that watched each lecture in the 2020 CVF course. The orange bar denotes the synchronous lectures (i.e., held live virtually). The green line represents the average of the 38 percentages generated across the lectures for the 2020 virtual course (80 ± 27%). (I) Average completion rate of each lecture for the 2019 in-person/hybrid course. The green line represents the average of each lecture’s completion rate for the 2019 hybrid in-person and recorded course (47 ± 16%). (J) Average completion rate of each lecture for the 2020 virtual course. The orange bar denotes the synchronous lectures (i.e., held live virtually). The green line represents the average of each lecture’s completion rate for the 2020 virtual course (86 ± 14%). CVF = coronavirus disease (COVID-19) virtual format; ID = identifier.

Figure 3.

Relationship between use of practice quizzes and examination grades. Students were sorted on the basis of the number of times they downloaded a quiz the same week it was uploaded. (A) Comparison of the number of on-time weekly quiz downloads and midterm scores for the 2019 hybrid in-person and recorded course. (B) Comparison of the number of on-time weekly quiz downloads before the midterm examination and midterm scores for the 2020 CVF course (P < 0.0001 for zero vs. one and P = 0.0037 for zero vs. two). (C) Comparison of the number of on-time weekly quiz downloads and final examination scores for the 2019 hybrid in-person and recorded course (P = 0.009 for zero vs. five). (D) Comparison of the number of on-time weekly quiz downloads and final examination score for the 2020 CVF course (P = 0.086). **P < 0.01 and ****P < 0.0001. CVF = coronavirus disease (COVID-19) virtual format.

Figure 4.

Relationship between use of task-oriented learning objectives (TOLOs) and examination grades. Students were sorted on the basis of the number of times they downloaded the TOLOs the same week they were uploaded. (A) Comparison of the number of on-time weekly TOLO downloads and final examination scores for the 2019 hybrid in-person and recorded course. (B) Comparison of the number of on-time weekly TOLO downloads and final examination scores for the 2020 coronavirus disease (COVID-19) virtual format course.