Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Sep:129:13-19.
doi: 10.1016/j.ijmedinf.2019.05.018. Epub 2019 May 23.

Automatic trial eligibility surveillance based on unstructured clinical data

Stéphane M Meystre  1 , Paul M Heider  2 , Youngjun Kim  2 , Daniel B Aruch  3 , Carolyn D Britten  3
Affiliations

Automatic trial eligibility surveillance based on unstructured clinical data

Stéphane M Meystre et al. Int J Med Inform. 2019 Sep.

Abstract

Introduction: Insufficient patient enrollment in clinical trials remains a serious and costly problem and is often considered the most critical issue to solve for the clinical trials community. In this project, we assessed the feasibility of automatically detecting a patient's eligibility for a sample of breast cancer clinical trials by mapping coded clinical trial eligibility criteria to the corresponding clinical information automatically extracted from text in the EHR.

Methods: Three open breast cancer clinical trials were selected by oncologists. Their eligibility criteria were manually abstracted from trial descriptions using the OHDSI ATLAS web application. Patients enrolled or screened for these trials were selected as 'positive' or 'possible' cases. Other patients diagnosed with breast cancer were selected as 'negative' cases. A selection of the clinical data and all clinical notes of these 229 selected patients was extracted from the MUSC clinical data warehouse and stored in a database implementing the OMOP common data model. Eligibility criteria were extracted from clinical notes using either manually crafted pattern matching (regular expressions) or a new natural language processing (NLP) application. These extracted criteria were then compared with reference criteria from trial descriptions. This comparison was realized with three different versions of a new application: rule-based, cosine similarity-based, and machine learning-based.

Results: For eligibility criteria extraction from clinical notes, the machine learning-based NLP application allowed for the highest accuracy with a micro-averaged recall of 90.9% and precision of 89.7%. For trial eligibility determination, the highest accuracy was reached by the machine learning-based approach with a per-trial AUC between 75.5% and 89.8%.

Conclusion: NLP can be used to extract eligibility criteria from EHR clinical notes and automatically discover patients possibly eligible for a clinical trial with good accuracy, which could be leveraged to reduce the workload of humans screening patients for trials.

Keywords: Clinical trial; Eligibility criteria; Machine learning; Natural language processing.

PubMed Disclaimer

Conflict of interest statement

COMPETING INTERESTS STATEMENT

The authors have no competing interests to declare.

Figures

Figure 1:
Figure 1:
WebAnno annotation tool (partial screenshot).
Figure 2:
Figure 2:
Clinical trial eligibility automatic surveillance process
Figure 3:
Figure 3:
Selected rows for representing a mention of “PR negative” in an OMOP CDM schema.
Figure 4:
Figure 4:
Features for SVM-based eligibility criteria extraction
See this image and copyright information in PMC

References

    1. Sung NS, Crowley WF, Genel M, Salber P, Sandy L, Sherwood LM, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003. March 12;289(10):1278–1287. - PubMed
    1. Lara PN, Higdon R, Lim N, Kwan K, Tanaka M, Lau DH, et al. Prospective evaluation of cancer clinical trial accrual patterns: identifying potential barriers to enrollment. J Clin Oncol 2001. March 15;19(6):1728–1733. - PubMed
    1. Somkin CP, Ackerson L, Husson G, Gomez V, Kolevska T, Goldstein D, et al. Effect of medical oncologists’ attitudes on accrual to clinical trials in a community setting. J Oncol Pract 2013. November;9(6):e275–83. - PMC - PubMed
    1. Penberthy L, Brown R, Puma F, Dahman B. Automated matching software for clinical trials eligibility: measuring efficiency and flexibility. Contemp Clin Trials 2010. May;31(3):207–217. - PMC - PubMed
    1. Thadani SR, Weng C, Bigger JT, Ennever JF, Wajngurt D. Electronic screening improves efficiency in clinical trial recruitment. Journal of the American Medical Informatics Association Oxford University Press; 2009. November;16(6):869–873. - PMC - PubMed

Publication types