Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Apr 11;20(4):e131.
doi: 10.2196/jmir.9085.

Evaluation of Technology-Enhanced Learning Programs for Health Care Professionals: Systematic Review

Pam Nicoll  1 , Sandra MacRury  2 , Hugo C van Woerden  2   3 , Keith Smyth  2
Affiliations

Evaluation of Technology-Enhanced Learning Programs for Health Care Professionals: Systematic Review

Pam Nicoll et al. J Med Internet Res. .

Abstract

Background: Technology-enhanced learning (TEL) programs are increasingly seen as the way in which education for health care professionals can be transformed, giving access to effective ongoing learning and training even where time or geographical barriers exist. Given the increasing emphasis on this mode of educational support for health care practitioners, it is vital that we can effectively evaluate and measure impact to ensure that TEL programs are effective and fit for purpose. This paper examines the current evidence base for the first time, in relation to the evaluation of TEL programs for health care professionals.

Objective: We conducted a systematic review of the current literature relating to the evaluation of TEL programs for health care professionals and critically appraised the quality of the studies.

Methods: This review employed specific search criteria to identify research studies that included evaluation of TEL for health care professionals. The databases searched included Medline Ovid, Cumulative Index of Nursing and Allied Health Literature Plus Advanced, Applied Social Sciences Index and Abstracts, ZETOC, Institute of Electrical and Electronics Engineers Explore Digital Library, Allied and Complementary Medicine, and Education Resources Information Center between January 2006 and January 2017. An additional hand search for relevant articles from reference lists was undertaken. Each of the studies identified was critically appraised for quality using the Crowe Critical Appraisal Tool. This approach produced a percentage total score for each study across specified categories. A proportion of the studies were independently assessed by an additional two reviewers.

Results: The review identified 21 studies that met the inclusion criteria. The studies included scored totals across eight categories within a range of 37%-95% and an average score of 68%. Studies that measured TEL using learner satisfaction surveys, or combined pretest and posttest knowledge score testing with learner satisfaction surveys, were found to be the most common types of TEL evaluations evident in the literature. The studies reviewed had low scores across reporting on ethical matters, design, and data collection categories.

Conclusions: There continues to be a need to develop effective and standard TEL evaluation tools, and good quality studies that describe effective evaluation of TEL education for health care professionals. Studies often fail to provide sufficient detail to support transferability or direct future TEL health care education programs.

Keywords: blended learning; digital learning; e-learning; effectiveness; evaluation; program evaluation; technology-enhanced learning.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Literature search exclusion chart.
See this image and copyright information in PMC

References

    1. Goodyear P, Retalis S. Technology-Enhanced Learning: Design Patterns and Pattern Language. Rotterdam, Netherlands: Sense Publishers; 2010. pp. 1–27.
    1. Kirkwood A, Price L. Technology-enhanced learning and teaching in higher education: what is ‘enhanced’ and how do we know? A critical literature review. Learn Media Technol. 2013 Feb 20;39(1):6–36. doi: 10.1080/17439884.2013.770404. - DOI
    1. Carroll C, Booth A, Papaioannou D, Sutton A, Wong R. UK health-care professionals' experience of on-line learning techniques: a systematic review of qualitative data. J Contin Educ Health Prof. 2009;29(4):235–41. doi: 10.1002/chp.20041. - DOI - PubMed
    1. Childs S, Blenkinsopp E, Hall A, Walton G. Effective e-learning for health professionals and students--barriers and their solutions. A systematic review of the literature--findings from the HeXL project. Health Info Libr J. 2005 Dec;22 Suppl 2:20–32. doi: 10.1111/j.1470-3327.2005.00614.x. doi: 10.1111/j.1470-3327.2005.00614.x.HIR614 - DOI - DOI - PubMed
    1. The Higher Education Funding Council for England . Digital Education Resource Archive (DERA) England: Higher Education Funding Council for England (HEFCE); 2009. [2018-03-12]. Enhancing learning and teaching through the use of technology: a revised approach to HEFCE’s strategy for e-learning http://dera.ioe.ac.uk/140/

Publication types