Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Dec 5:2018:1046-1055.
eCollection 2018.

Selecting Test Cases from the Electronic Health Record for Software Testing of Knowledge-Based Clinical Decision Support Systems

Omar A Usman  1   2 , Connie Oshiro  1 , Justin G Chambers  1 , Samson W Tu  2 , Susana Martins  1 , Amy Robinson  1 , Mary K Goldstein  1   2
Affiliations

Selecting Test Cases from the Electronic Health Record for Software Testing of Knowledge-Based Clinical Decision Support Systems

Omar A Usman et al. AMIA Annu Symp Proc. .

Abstract

Software testing of knowledge-based clinical decision support systems is challenging, labor intensive, and expensive; yet, testing is necessary since clinical applications have heightened consequences. Thoughtful test case selection improves testing coverage while minimizing testing burden. ATHENA-CDS is a knowledge-based system that provides guideline-based recommendations for chronic medical conditions. Using the ATHENA-CDS diabetes knowledgebase, we demonstrate a generalizable approach for selecting test cases using rules/ filters to create a set of paths that mimics the system's logic. Test cases are allocated to paths using a proportion heuristic. Using data from the electronic health record, we found 1,086 cases with glycemic control above target goals. We created a total of 48 filters and 50 unique system paths, which were used to allocate 200 test cases. We show that our method generates a comprehensive set of test cases that provides adequate coverage for the testing of a knowledge-based CDS.

Keywords: Clinical Decision Support; Expert Systems; Knowledge-Based Systems; Offline Testing; Path Testing; Software Testing; Software Verification; System Testing; Test Case Selection.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Test Case Selection and System Verification Process
Figure 2.
Figure 2.
Initial Path Tree Diagram
Figure 3.
Figure 3.
Drug Scenarios and Endpoints
Figure 4.
Figure 4.
Proportion of Test Cases
See this image and copyright information in PMC

References

    1. Plant RT. Rigorous approach to the development of knowledge-based systems. Knowl-Based Syst. 1991 Dec 1;4(4):186–96.
    1. Goldstein MK, Hoffman BB, Coleman RW, Musen MA, Tu SW, Advani A, et al. Proc AMIA Symp. 2000. Implementing clinical practice guidelines while taking account of changing evidence: ATHENA DSS, an easily modifiable decision-support system for managing hypertension in primary care; pp. 300–4. - PMC - PubMed
    1. Goldstein MK, Hoffman BB, Coleman RW, Tu SW, Shankar RD, O’Connor M, et al. Proc AMIA Symp. 2001. Patient safety in guideline-based decision support for hypertension management: ATHENA DSS; pp. 214–8. - PMC - PubMed
    1. Martins SB, Lai S, Tu S, Shankar R, Hastings SN, Hoffman BB, et al. AMIA Annu Symp Proc. 2006. Offline testing of the ATHENA Hypertension decision support system knowledge base to improve the accuracy of recommendations; pp. 539–43. - PMC - PubMed
    1. Tso GJ, Yuen K, Martins S, Tu SW, Ashcraft M, Heidenreich P, et al. Test Case Selection in Pre-Deployment Testing of Complex Clinical Decision Support Systems. AMIA Summits Transl Sci Proc. 2016;2016:240–9. - PMC - PubMed

Publication types

Substances

LinkOut - more resources