Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems--I-MeDeSA

Abstract

Background: Medication-related decision support can reduce the frequency of preventable adverse drug events. However, the design of current medication alerts often results in alert fatigue and high over-ride rates, thus reducing any potential benefits.

Methods: The authors previously reviewed human-factors principles for relevance to medication-related decision support alerts. In this study, instrument items were developed for assessing the appropriate implementation of these human-factors principles in drug-drug interaction (DDI) alerts. User feedback regarding nine electronic medical records was considered during the development process. Content validity, construct validity through correlation analysis, and inter-rater reliability were assessed.

Results: The final version of the instrument included 26 items associated with nine human-factors principles. Content validation on three systems resulted in the addition of one principle (Corrective Actions) to the instrument and the elimination of eight items. Additionally, the wording of eight items was altered. Correlation analysis suggests a direct relationship between system age and performance of DDI alerts (p=0.0016). Inter-rater reliability indicated substantial agreement between raters (κ=0.764).

Conclusion: The authors developed and gathered preliminary evidence for the validity of an instrument that measures the appropriate use of human-factors principles in the design and display of DDI alerts. Designers of DDI alerts may use the instrument to improve usability and increase user acceptance of medication alerts, and organizations selecting an electronic medical record may find the instrument helpful in meeting their clinicians' usability needs.

Figure 1

Search criteria and results. (A) (Ambulatory Care Information Systems OR Computerized Order Entry OR Computerized Physician Order Entry OR Computerized Prescriber Order Entry OR Computerized Provider Order Entry OR Decision Support Systems, Clinical OR Drug Therapy, Computer-Assisted OR Electronic Medical Record OR Electronic Order Entry OR Electronic Physician Order Entry OR Electronic Prescribing OR Electronic Prescription OR Medical Order Entry Systems OR Medical Records Systems, Computerized). (B) (Interface, User Computer OR Interface, User–Computer OR Interfaces, User–Computer OR Human Factor OR Human Factors OR Human Factors Engineering OR Human Factors Principles OR Human–Machine Interaction OR Human–Machine Interface Usability OR Interfaces, User Computer OR Usability OR User Computer Interface OR User Computer Interfaces OR User–Computer Interfaces OR User Interface Design). (C) (Instrument* OR Methodology, Survey OR Methods, Survey OR Questionnaire* OR Survey* OR Survey Method OR Survey Methodology). (D) (Limits: Publication Types: Clinical Trial; Meta-Analysis; Practice Guideline; Randomized Controlled Trial; Review; Articles with Abstracts; Language: English; Publication Date Range: July 2000–July 2010).