Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Sep 6;8(3):910-923.
doi: 10.4338/ACI-2017-01-RA-0006.

Clinical decisions support malfunctions in a commercial electronic health record

Steven Z Kassakian, Thomas R Yackel, Paul N Gorman, David A Dorr

Clinical decisions support malfunctions in a commercial electronic health record

Steven Z Kassakian et al. Appl Clin Inform. .

Abstract

Objectives: Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.

Methods: We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.

Results: Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.

Discussion: CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.

Conclusion: CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.

Keywords: Clinical decision support; alerts; electronic health record; electronic medical record; errors; malfunction.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest The authors acknowledge they have no conflicts of interest in the research.

Figures

Fig. 1
Fig. 1
Alert activity from CDS rule related use of enoxaparin order set, determined to be a malfunction
Fig. 2
Fig. 2
Alert activity from CDS rule related to influenza administration prior to discharge, determined to be a malfunction
Fig. 3
Fig. 3
Alert activity from CDS rule related to height documentation in oncology clinic, determined to be a malfunction
Fig. 4
Fig. 4
Alert activity from CDS rule related to coronary artery disease and appropriate medications (RXs), determined to be a malfunction.
See this image and copyright information in PMC

References

    1. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D. Effect of clinical decision-support systems: a systematic review. Annals of internal medicine. 2012;157(01):29–43. - PubMed
    1. Lobach D. Enabling Health Care Decision-making Through Clinical Decision Support and Knowledge Management. 2012. - PMC - PubMed
    1. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. The New England journal of medicine. 1976;295(24):1351–5. - PubMed
    1. Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, Rigon G, Vaona A, Ruggiero F, Mangia M, Iorio A, Kunnamo I, Bonovas S. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health. 2014;104(12):e12–22. - PMC - PubMed
    1. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, Garg AX, Haynes RB.Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trialsBmj. 2013;346:f657. doi: 10.1136/bmj.f657. PubMed PMID: 23412440. - PubMed

Substances