Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Jan;24(1):218-226.
doi: 10.1093/jamia/ocw046. Epub 2016 Apr 23.

An appraisal of published usability evaluations of electronic health records via systematic review

Marc A Ellsworth  1   2 , Mikhail Dziadzko  2   3 , John C O'Horo  2   4 , Ann M Farrell  5 , Jiajie Zhang  6 , Vitaly Herasevich  2   3
Affiliations

An appraisal of published usability evaluations of electronic health records via systematic review

Marc A Ellsworth et al. J Am Med Inform Assoc. 2017 Jan.

Abstract

Objective: In this systematic review, we aimed to evaluate methodological and reporting trends present in the current literature by investigating published usability studies of electronic health records (EHRs).

Methods: A literature search was conducted for articles published through January 2015 using MEDLINE (Ovid), EMBASE, Scopus, and Web of Science, supplemented by citation and reference list reviews. Studies were included if they tested the usability of hospital and clinic EHR systems in the inpatient, outpatient, emergency department, or operating room setting.

Results: A total of 4848 references were identified for title and abstract screening. Full text screening was performed for 197 articles, with 120 meeting the criteria for study inclusion.

Conclusion: A review of the literature demonstrates a paucity of quality published studies describing scientifically valid and reproducible usability evaluations at various stages of EHR system development. A lack of formal and standardized reporting of EHR usability evaluation results is a major contributor to this knowledge gap, and efforts to improve this deficiency will be one step of moving the field of usability engineering forward.

Keywords: electronic health records; health information technology; human factors; usability.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Study selection flow chart.
Figure 2.
Figure 2.
Usability method distribution by type of evaluation. R/D, requirements/development; I/PI, implementation/post-implementation.
Figure 3.
Figure 3.
Distribution of included studies, stratified by year. The number of studies that were published within each 5-year increment is in parentheses.
See this image and copyright information in PMC

References

    1. Lowry SZ, Quinn MT, Ramaiah M, et al. Technical evaluation, testing, and validation of the usability of electronic health records. US Department of Commerce, National Institute of Standards and Technology (NIST). NIST Interagency/Internal Report (NISTIR) – 7804. 2012.
    1. Shumacher RM, Lowry SZ. NIST guide to the processes approach for improving the usability of electronic health records. US Department of Commerce, National Institute of Standards and Technology (NIST). NIST Interagency/Internal Report (NISTIR) – 7741. 2010.
    1. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116(6):1506–1512. - PubMed
    1. US Department of Health and Human Services (HHS) Office of the Assistant Secretary for Public Affairs, Digital Communications Division. Usability.gov. http://www.usability.gov. Accessed May 1, 2015.
    1. US Department of Health and Human Services (HHS), Office of the National Coordinator for Health Information Technology. Health Information Technology: Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology. - PubMed

Publication types