Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Jun 18;5(2):571-88.
doi: 10.4338/ACI-2014-01-RA-0005. eCollection 2014.

Evaluation of a Korean version of a tool for assessing the incorporation of human factors into a medication-related decision support system: the I-MeDeSA

I Cho  1 , J Lee  2 , H Han  3 , S Phansalkar  4 , D W Bates  5
Affiliations

Evaluation of a Korean version of a tool for assessing the incorporation of human factors into a medication-related decision support system: the I-MeDeSA

I Cho et al. Appl Clin Inform. .

Abstract

Objective: The Instrument for Evaluating Human-Factor Principles in Medication-Related Decision Support Alerts (I-MeDeSA) was developed recently in the US with a view towards improving considerations of human-factor principles when designing alerts for clinical decision support (CDS) systems. This study evaluated the generalizability of this tool, in cooperation with its authors, across cultures by applying it to a Korean system. We also examined opportunities to promote user acceptance of the system.

Methods: We developed a Korean version of the I-MeDeSA (K-I-MeDeSA) and used it to evaluate drug-drug interaction alerts in a large academic tertiary hospital in Seoul. We involved four reviewers (A, B, C, and D). Two (A and B) conducted the initial independent scoring, while the other two (C and D) performed a final review and assessed feedback from the initial reviewers. The obtained scores were compared with those from 13 previously reported CDS systems. The feedback was summarized qualitatively.

Results: The translation of the I-MeDeSA had excellent interrater agreement in terms of face validity (scale-level content validity index = 0.95). The system's K-I-MeDeSA score was 10 out of 26, with a good agreement between reviewers (κ = 0.77), which showed a lack of human-factor considerations. The reviewers readily identified two of the nine principles that needed primary improvement: prioritization and text-based information. The reviewers also expressed difficulty judging the following four principles: alarm philosophy, visibility, color, and learnability and confusability.

Conclusion: The K-I-MeDeSA was semantically and operationally equivalent to the original tool. Only minor cultural problems were identified, leading the reviewers to suggest the need for clarification of certain words plus a more detailed description of the tool's rationale and exemplars. Further evaluation is needed to empirically assess whether the implementation of changes in an electronic health record system could improve the adoption of CDS alerts.

Keywords: Clinical decision support; alert fatigue; computerized order entry; human-computer interaction; interfaces and usability.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest

None of the authors has any conflicts of interest.

Figures

Fig. 1
Fig. 1
Example screenshot of the redesigned medication-related alert including the DDIs in the EHR. (a) Several different types of alert stratified according to group. (b) An DDI alert.
Fig. 2
Fig. 2
Sample screenshots of DDI alerts from the highest-scoring EHR system.
Fig. 3
Fig. 3
Comparison of the evaluation results with other in-house and commercial alert systems (the systems are ordered according to score and not in the order they are listed in Table 2)
See this image and copyright information in PMC

References

    1. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS.Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med 2007; 167(13):1400–1405 - PubMed
    1. Bates DW, Gawande AA.Improving safety with information technology. N Engl J Med 2003; 348(25):2526–24534 - PubMed
    1. Fischer MA, Solomon DH, Teich JM, Avorn J.Conversion from intravenous to oral medications: Assessment of a computerized intervention for hospitalized patients. Arch Intern Med 2003; 163(21):2585–259 - PubMed
    1. Wang SJ, Middleton B, Prosser LA, Bardon CG, Spurr CD, Carchidi PJ, Kittler AF, Goldszer RC, Fairchild DG, Sussman AJ, Kuperman GJ, Bates DW.A cost-benefit analysis of electronic medical records in primary care. Am J Med 2003; 114(5):397–403 - PubMed
    1. Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD.Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med 2003; 18(12):967–976 - PMC - PubMed

Publication types