What Are We Looking for in Computer-Based Learning Interventions in Medical Education? A Systematic Review

Abstract

Background: Computer-based learning (CBL) has been widely used in medical education, and reports regarding its usage and effectiveness have ranged broadly. Most work has been done on the effectiveness of CBL approaches versus traditional methods, and little has been done on the comparative effects of CBL versus CBL methodologies. These findings urged other authors to recommend such studies in hopes of improving knowledge about which CBL methods work best in which settings.

Objective: In this systematic review, we aimed to characterize recent studies of the development of software platforms and interventions in medical education, search for common points among studies, and assess whether recommendations for CBL research are being taken into consideration.

Methods: We conducted a systematic review of the literature published from 2003 through 2013. We included studies written in English, specifically in medical education, regarding either the development of instructional software or interventions using instructional software, during training or practice, that reported learner attitudes, satisfaction, knowledge, skills, or software usage. We conducted 2 latent class analyses to group articles according to platform features and intervention characteristics. In addition, we analyzed references and citations for abstracted articles.

Results: We analyzed 251 articles. The number of publications rose over time, and they encompassed most medical disciplines, learning settings, and training levels, totaling 25 different platforms specifically for medical education. We uncovered 4 latent classes for educational software, characteristically making use of multimedia (115/251, 45.8%), text (64/251, 25.5%), Web conferencing (54/251, 21.5%), and instructional design principles (18/251, 7.2%). We found 3 classes for intervention outcomes: knowledge and attitudes (175/212, 82.6%), knowledge, attitudes, and skills (11.8%), and online activity (12/212, 5.7%). About a quarter of the articles (58/227, 25.6%) did not hold references or citations in common with other articles. The number of common references and citations increased in articles reporting instructional design principles (P=.03), articles measuring online activities (P=.01), and articles citing a review by Cook and colleagues on CBL (P=.04). There was an association between number of citations and studies comparing CBL versus CBL, independent of publication date (P=.02).

Conclusions: Studies in this field vary highly, and a high number of software systems are being developed. It seems that past recommendations regarding CBL interventions are being taken into consideration. A move into a more student-centered model, a focus on implementing reusable software platforms for specific learning contexts, and the analysis of online activity to track and predict outcomes are relevant areas for future research in this field.

Keywords: b-learning; computer-based learning; e-learning; internet-based learning; medical education; systematic review.

Figure 1

Flow of a systematic review of the literature published January 1, 2003 to December 31, 2013 regarding either the development of instructional software or interventions using instructional software.

Figure 2

Articles published per country of medical school. The article count axis is presented in logarithmic scale for better data representation.

Figure 3

Articles per basic science and clinical subject. The article count axis is presented in logarithmic scale for better data representation.

Figure 4

Prevalence of articles per educational software feature and educational software latent class. Horizontal axis ranges between 0 and 100 on a squared root scale. Point color specifies the probability of assigning a paper to each class based on the presence of each variable (gray indicates a probability of 0, ranging to dark blue indicating the highest probability). From the listed variables, those present in more than 2% of all articles were used to determine the educational software latent classes.

Figure 5

Prevalence of articles per intervention feature and intervention endpoint latent class. Horizontal axis ranges between 0 and 100 on a squared root scale. Point color specifies the probability of assigning a paper to each class based on the presence of each variable (gray indicates a probability of 0, ranging to dark blue indicating the highest probability). Only variables regarding assessment of knowledge, attitudes, skills, and online activity (the 4 last panels) were used to determine intervention end point latent classes. CBL: computer-based learning.

Figure 6

Relationships between articles included in this review (indicated by nodes). Links between nodes indicate that articles have references and citations in common. The width of the link indicates the number of studies in common, ranging from 1 to 5. About a quarter of the studies have no common references or citations. Only 227 of the 251 studies were included in this analysis due to missing information (90.4%).

Figure 7

Mean citation number differences between traditional versus computer-based learning (CBL), and CBL versus CBL, adjusted for publication date. For CBL versus CBL, only 227 of the 251 studies were included in this analysis due to missing information (90.4%). Error bars represent the 95% CI.

Figure 8

Mean number of related articles per latent class and reference to the Cook et al review. Number of related articles is adjusted for publication date. P values indicate intraclass pairwise differences from the topmost element of each color-coded class. Significant relationships are marked in bold typeface. Only 227 of the 251 studies were included in this analysis due to missing information (90.4%). Error bars represent the 95% CI.