Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Jul 16;7(3):e13331.
doi: 10.2196/13331.

Improving the Efficacy of the Data Entry Process for Clinical Research With a Natural Language Processing-Driven Medical Information Extraction System: Quantitative Field Research

Jiang Han #  1   2 , Ken Chen #  3 , Lei Fang  3 , Shaodian Zhang  3   4 , Fei Wang  3   5 , Handong Ma  3   6 , Liebin Zhao  2   7 , Shijian Liu  1   2
Affiliations

Improving the Efficacy of the Data Entry Process for Clinical Research With a Natural Language Processing-Driven Medical Information Extraction System: Quantitative Field Research

Jiang Han et al. JMIR Med Inform. .

Abstract

Background: The growing interest in observational trials using patient data from electronic medical records poses challenges to both efficiency and quality of clinical data collection and management. Even with the help of electronic data capture systems and electronic case report forms (eCRFs), the manual data entry process followed by chart review is still time consuming.

Objective: To facilitate the data entry process, we developed a natural language processing-driven medical information extraction system (NLP-MIES) based on the i2b2 reference standard. We aimed to evaluate whether the NLP-MIES-based eCRF application could improve the accuracy and efficiency of the data entry process.

Methods: We conducted a randomized and controlled field experiment, and 24 eligible participants were recruited (12 for the manual group and 12 for NLP-MIES-supported group). We simulated the real-world eCRF completion process using our system and compared the performance of data entry on two research topics, pediatric congenital heart disease and pneumonia.

Results: For the congenital heart disease condition, the NLP-MIES-supported group increased accuracy by 15% (95% CI 4%-120%, P=.03) and reduced elapsed time by 33% (95% CI 22%-42%, P<.001) compared with the manual group. For the pneumonia condition, the NLP-MIES-supported group increased accuracy by 18% (95% CI 6%-32%, P=.008) and reduced elapsed time by 31% (95% CI 19%-41%, P<.001).

Conclusions: Our system could improve both the accuracy and efficiency of the data entry process.

Keywords: case report form; electric medical records; electronic data capture; field research; natural language processing.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Workflow of the natural language processing–driven medical information extraction system. EMR: electronic medical record; NLP: natural language processing; eCRF: electronic case report form.
Figure 2
Figure 2
Electronic case report form design for congenital heart disease.
Figure 3
Figure 3
Electronic case report form design for pneumonia.
Figure 4
Figure 4
Graphic user interface for electronic case report form (eCRF) data entry.
See this image and copyright information in PMC

References

    1. ClinicalTrials.gov. [2019-06-12]. Trends, charts, and maps https://clinicaltrials.gov/ct2/resources/trends .
    1. Walther B, Hossin S, Townend J, Abernethy N, Parker D, Jeffries D. Comparison of electronic data capture (EDC) with the standard data capture method for clinical trial data. PLoS One. 2011;6(9):e25348. doi: 10.1371/journal.pone.0025348. http://dx.plos.org/10.1371/journal.pone.0025348 PONE-D-11-05243 - DOI - DOI - PMC - PubMed
    1. Bellary S, Krishnankutty B, Latha MS. Basics of case report form designing in clinical research. Perspect Clin Res. 2014 Oct;5(4):159–166. doi: 10.4103/2229-3485.140555. http://www.picronline.org/article.asp?issn=2229-3485;year=2014;volume=5;... PCR-5-159 - DOI - PMC - PubMed
    1. Fleischmann R, Decker A, Kraft A, Mai K, Schmidt S. Mobile electronic versus paper case report forms in clinical trials: a randomized controlled trial. BMC Med Res Methodol. 2017 Dec 01;17(1):153. doi: 10.1186/s12874-017-0429-y. https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-017-... 10.1186/s12874-017-0429-y - DOI - DOI - PMC - PubMed
    1. Dillon DG, Pirie F, Rice S, Pomilla C, Sandhu MS, Motala AA, Young EH, African Partnership for Chronic Disease Research (APCDR) Open-source electronic data capture system offered increased accuracy and cost-effectiveness compared with paper methods in Africa. J Clin Epidemiol. 2014 Dec;67(12):1358–1363. doi: 10.1016/j.jclinepi.2014.06.012. https://linkinghub.elsevier.com/retrieve/pii/S0895-4356(14)00238-8 S0895-4356(14)00238-8 - DOI - PMC - PubMed