Evaluation of medication alerts in electronic health records for compliance with human factors principles

Abstract

Introduction: Increasing the adoption of electronic health records (EHRs) with integrated clinical decision support (CDS) is a key initiative of the current US healthcare administration. High over-ride rates of CDS alerts strongly limit these potential benefits. As a result, EHR designers aspire to improve alert design to achieve better acceptance rates. In this study, we evaluated drug-drug interaction (DDI) alerts generated in EHRs and compared them for compliance with human factors principles.

Methods: We utilized a previously validated questionnaire, the I-MeDeSA, to assess compliance with nine human factors principles of DDI alerts generated in 14 EHRs. Two reviewers independently assigned scores evaluating the human factors characteristics of each EHR. Rankings were assigned based on these scores and recommendations for appropriate alert design were derived.

Results: The 14 EHRs evaluated in this study received scores ranging from 8 to 18.33, with a maximum possible score of 26. Cohen's κ (κ=0.86) reflected excellent agreement among reviewers. The six vendor products tied for second and third place rankings, while the top system and bottom five systems were home-grown products. The most common weaknesses included the absence of characteristics such as alert prioritization, clear and concise alert messages indicating interacting drugs, actions for clinical management, and a statement indicating the consequences of over-riding the alert.

Conclusions: We provided detailed analyses of the human factors principles which were assessed and described our recommendations for effective alert design. Future studies should assess whether adherence to these recommendations can improve alert acceptance.

Keywords: Clinical Decision Support; Drug Drug Interaction; EHRs; Electronic Health Records; Human Factors; Medication-Related Decision Support.

Figure 1

Example of a system that scored highly on the construct of Placement by identifying the type of interaction, allowing the user to easily enter in their response to the alert, linking the alert to the medication order by appropriate timing, and providing the critical information needed to act on the drug-drug interaction alert.

Figure 2

The drug–drug interaction alert presented here shows insufficient information for the user to act on the interaction between warfarin and the interacting drugs on the patient's medication profile. In addition, reviewers found it difficult to read the statement indicating the interacting drugs in bright yellow font on a dark blue background.

Figure 3

(A–C) Illustration of the Prioritization principle. (A) Use of color-coding for distinction between alert severities (B) and (C). Use of symbols to indicate appropriate severity levels.

Figure 4

System 12 scored poorly on Learnability and confusability because it did not present unique visual characteristics for differentiating between alert severities.

Figure 5

Interacting drugs, management steps, and potential consequence to the patient clearly presented when an alert is displayed.

Figure 6

Poor corrective actions do not allow the user to provide a response to an alert. In addition, this system utilized more than 10 colors on the screen.