Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2008 Aug;23(8):1164-71.
doi: 10.1007/s11606-008-0604-2. Epub 2008 Apr 30.

Knowledge retention after an online tutorial: a randomized educational experiment among resident physicians

Douglas S Bell  1 , Charles E Harless, Jerilyn K Higa, Elizabeth L Bjork, Robert A Bjork, Mohsen Bazargan, Carol M Mangione
Affiliations
Randomized Controlled Trial

Knowledge retention after an online tutorial: a randomized educational experiment among resident physicians

Douglas S Bell et al. J Gen Intern Med. 2008 Aug.

Abstract

Background: The time course of physicians' knowledge retention after learning activities has not been well characterized. Understanding the time course of retention is critical to optimizing the reinforcement of knowledge.

Design: Educational follow-up experiment with knowledge retention measured at 1 of 6 randomly assigned time intervals (0-55 days) after an online tutorial covering 2 American Diabetes Association guidelines.

Participants: Internal and family medicine residents.

Measurements: Multiple-choice knowledge tests, subject characteristics including critical appraisal skills, and learner satisfaction.

Results: Of 197 residents invited, 91 (46%) completed the tutorial and were randomized; of these, 87 (96%) provided complete follow-up data. Ninety-two percent of the subjects rated the tutorial as "very good" or "excellent." Mean knowledge scores increased from 50% before the tutorial to 76% among those tested immediately afterward. Score gains were only half as great at 3-8 days and no significant retention was measurable at 55 days. The shape of the retention curve corresponded with a 1/4-power transformation of the delay interval. In multivariate analyses, critical appraisal skills and participant age were associated with greater initial learning, but no participant characteristic significantly modified the rate of decline in retention.

Conclusions: Education that appears successful from immediate posttests and learner evaluations can result in knowledge that is mostly lost to recall over the ensuing days and weeks. To achieve longer-term retention, physicians should review or otherwise reinforce new learning after as little as 1 week.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Disposition of all individuals eligible for study participation through recruitment, tutorial participation, randomization, posttest, and analysis.
Figure 2
Figure 2
Chart of knowledge test results for each time interval after the tutorial with squares representing the mean scores on the final 17-item posttest and circles representing the sum of scores on the posttest for the 3 malfunctioning items. Horizontal dotted lines represent subjects’ mean knowledge scores before the tutorial for each of the 2 item groups. Vertical bars show the standard error of the mean. Solid lines represent the best linear fit for each outcome versus the time delay, after fourth-root transformation of the time interval. (17-item posttest score = 13.3–1.63t1/4, R2 = 0.27, P < .0001; posttest sum of 3 malfunctioning items = 1.26 + 0.23t1/4, R2 = 0.05, P = .04.).
See this image and copyright information in PMC

References

    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'DOI', 'value': '10.1001/jama.282.9.867', 'is_inner': False, 'url': 'https://doi.org/10.1001/jama.282.9.867'}, {'type': 'PubMed', 'value': '10478694', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/10478694/'}]}
    2. Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P,Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867–74. - PubMed
    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'PubMed', 'value': '15710959', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/15710959/'}]}
    2. Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005;142(4):260–73. - PubMed
    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'PubMed', 'value': '874991', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/874991/'}]}
    2. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ. 1977;52(7):578–81. - PubMed
    1. {'text': '', 'ref_index': 1, 'ids': [{'type': 'DOI', 'value': '10.1001/jama.255.4.501', 'is_inner': False, 'url': 'https://doi.org/10.1001/jama.255.4.501'}, {'type': 'PubMed', 'value': '3941533', 'is_inner': True, 'url': 'https://pubmed.ncbi.nlm.nih.gov/3941533/'}]}
    2. Evans CE, Haynes RB, Birkett NJ, Gilbert JR, Taylor DW, Sackett DL, et al. Does a mailed continuing education program improve physician performance? Results of a randomized trial in antihypertensive care. JAMA. 1986;255(4):501–4. - PubMed
    1. Marinopoulos SS, Dorman T, Ratanawongsa N, Wilson LM, Ashar BH, Magaziner JL, et al. Effectiveness of Continuing Medical Education. Rockville, MD: Agency for Healthcare Research and Quality; 2007. Evidence Report AHRQ Publication No. 07-E006, January.

Publication types