Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Nov;23(6):1068-1076.
doi: 10.1093/jamia/ocw005. Epub 2016 Mar 28.

Analysis of clinical decision support system malfunctions: a case series and survey

Adam Wright  1   2   3 , Thu-Trang T Hickman  4 , Dustin McEvoy  3 , Skye Aaron  4 , Angela Ai  4 , Jan Marie Andersen  4 , Salman Hussain  4   5 , Rachel Ramoni  2   6 , Julie Fiskio  4 , Dean F Sittig  7 , David W Bates  4   2   3
Affiliations

Analysis of clinical decision support system malfunctions: a case series and survey

Adam Wright et al. J Am Med Inform Assoc. 2016 Nov.

Abstract

Objective: To illustrate ways in which clinical decision support systems (CDSSs) malfunction and identify patterns of such malfunctions.

Materials and methods: We identified and investigated several CDSS malfunctions at Brigham and Women's Hospital and present them as a case series. We also conducted a preliminary survey of Chief Medical Information Officers to assess the frequency of such malfunctions.

Results: We identified four CDSS malfunctions at Brigham and Women's Hospital: (1) an alert for monitoring thyroid function in patients receiving amiodarone stopped working when an internal identifier for amiodarone was changed in another system; (2) an alert for lead screening for children stopped working when the rule was inadvertently edited; (3) a software upgrade of the electronic health record software caused numerous spurious alerts to fire; and (4) a malfunction in an external drug classification system caused an alert to inappropriately suggest antiplatelet drugs, such as aspirin, for patients already taking one. We found that 93% of the Chief Medical Information Officers who responded to our survey had experienced at least one CDSS malfunction, and two-thirds experienced malfunctions at least annually.

Discussion: CDSS malfunctions are widespread and often persist for long periods. The failure of alerts to fire is particularly difficult to detect. A range of causes, including changes in codes and fields, software upgrades, inadvertent disabling or editing of rules, and malfunctions of external systems commonly contribute to CDSS malfunctions, and current approaches for preventing and detecting such malfunctions are inadequate.

Conclusion: CDSS malfunctions occur commonly and often go undetected. Better methods are needed to prevent and detect these malfunctions.

Keywords: anomaly detection; clinical decision support; electronic health records; machine learning; safety.

PubMed Disclaimer

Figures

Figure 1:
Figure 1:
Laboratory monitoring reminders for amiodarone in the Partners Healthcare longitudinal medical record (LMR). The main screen of the LMR is shown in the background, with the reminders enlarged and the amiodarone reminders highlighted in a box.
Figure 2:
Figure 2:
Pseudocode representation of the amiodarone/thyroid-stimulating hormone (TSH) test reminder.
Figure 3:
Figure 3:
Firing rate of four alerts at Brigham and Women’s Hospital over a 5-year period (weekend days are represented by darker dots, and weekdays are represented by lighter dots), with anomalies indicated (superimposed horizontal bars show anomalous periods).
Figure 4:
Figure 4:
Sample reminder display for a 2-month-old boy, containing numerous inappropriate reminders for this patient.
See this image and copyright information in PMC

References

    1. Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs (Project Hope). 2011;30(3):464–471. - PubMed
    1. Lyman JA, Cohn WF, Bloomrosen M, Detmer DE. Clinical decision support: progress and opportunities. J Am Med Inform Assoc. 2010;17(5):487–492. - PMC - PubMed
    1. Kawamoto K, Lobach DF. Proposal for fulfilling strategic objectives of the U.S. Roadmap for national action on clinical decision support through a service-oriented architecture leveraging HL7 services. J Am Med Inform Assoc. 2007;14(2):146–155. - PMC - PubMed
    1. Garg AX, Adhikari NKJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes. JAMA. 2005;293(10):1223–1238. - PubMed
    1. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765. - PMC - PubMed

Publication types