Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Nov 7;20(11):e10497.
doi: 10.2196/10497.

Automated Extraction of Diagnostic Criteria From Electronic Health Records for Autism Spectrum Disorders: Development, Evaluation, and Application

Gondy Leroy  1 , Yang Gu  1 , Sydney Pettygrove  1 , Maureen K Galindo  1 , Ananyaa Arora  1 , Margaret Kurzius-Spencer  1
Affiliations

Automated Extraction of Diagnostic Criteria From Electronic Health Records for Autism Spectrum Disorders: Development, Evaluation, and Application

Gondy Leroy et al. J Med Internet Res. .

Abstract

Background: Electronic health records (EHRs) bring many opportunities for information utilization. One such use is the surveillance conducted by the Centers for Disease Control and Prevention to track cases of autism spectrum disorder (ASD). This process currently comprises manual collection and review of EHRs of 4- and 8-year old children in 11 US states for the presence of ASD criteria. The work is time-consuming and expensive.

Objective: Our objective was to automatically extract from EHRs the description of behaviors noted by the clinicians in evidence of the diagnostic criteria in the Diagnostic and Statistical Manual of Mental Disorders (DSM). Previously, we reported on the classification of entire EHRs as ASD or not. In this work, we focus on the extraction of individual expressions of the different ASD criteria in the text. We intend to facilitate large-scale surveillance efforts for ASD and support analysis of changes over time as well as enable integration with other relevant data.

Methods: We developed a natural language processing (NLP) parser to extract expressions of 12 DSM criteria using 104 patterns and 92 lexicons (1787 terms). The parser is rule-based to enable precise extraction of the entities from the text. The entities themselves are encompassed in the EHRs as very diverse expressions of the diagnostic criteria written by different people at different times (clinicians, speech pathologists, among others). Due to the sparsity of the data, a rule-based approach is best suited until larger datasets can be generated for machine learning algorithms.

Results: We evaluated our rule-based parser and compared it with a machine learning baseline (decision tree). Using a test set of 6636 sentences (50 EHRs), we found that our parser achieved 76% precision, 43% recall (ie, sensitivity), and >99% specificity for criterion extraction. The performance was better for the rule-based approach than for the machine learning baseline (60% precision and 30% recall). For some individual criteria, precision was as high as 97% and recall 57%. Since precision was very high, we were assured that criteria were rarely assigned incorrectly, and our numbers presented a lower bound of their presence in EHRs. We then conducted a case study and parsed 4480 new EHRs covering 10 years of surveillance records from the Arizona Developmental Disabilities Surveillance Program. The social criteria (A1 criteria) showed the biggest change over the years. The communication criteria (A2 criteria) did not distinguish the ASD from the non-ASD records. Among behaviors and interests criteria (A3 criteria), 1 (A3b) was present with much greater frequency in the ASD than in the non-ASD EHRs.

Conclusions: Our results demonstrate that NLP can support large-scale analysis useful for ASD surveillance and research. In the future, we intend to facilitate detailed analysis and integration of national datasets.

Keywords: Autism Spectrum Disorder; DSM; complex entity extraction; decision tree; electronic health records; machine learning; natural language processing; parser.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Visualization of 2 (of 7 existing) patterns for Diagnostic Manual of Mental Disorders criteria A2c.
Figure 2
Figure 2
Descriptive information on 4480 records available electronically from the Arizona Developmental Disabilities Surveillance Program.
Figure 3
Figure 3
Electronic health record word count for autism spectrum disorder (ASD) and non-ASD cases.
Figure 4
Figure 4
Average A1 criteria per record. ASD: autism spectrum disorder; EHR: electronic health record.
Figure 5
Figure 5
Average A2 criteria per record. ASD: autism spectrum disorder; EHR: electronic health record.
Figure 6
Figure 6
Average A3 criteria per record. ASD: autism spectrum disorder; EHR: electronic health record.
See this image and copyright information in PMC

References

    1. Christensen DL, Baio J, Van Naarden Braun K, Bilder D, Charles J, Constantino JN, Daniels J, Durkin MS, Fitzgerald RT, Kurzius-Spencer M, Lee LC, Pettygrove S, Robinson C, Schulz E, Wells C, Wingate MS, Zahorodny W, Yeargin-Allsopp M. Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2012. Surveillance Summaries. 2016 Apr;65(3):1–23. doi: 10.15585/mmwr.ss6503a1. - DOI - PMC - PubMed
    1. Abney S, Schapire RE, Singer Y. Boosting Applied to Tagging and PP Attachment. Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora; Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora; June 21-22; College Park, MD, USA. 1999. https://aclanthology.coli.uni-saarland.de/events/emnlp-1999
    1. Principal Investigators Prevalence of Autism Spectrum Disorders --- Autism and Developmental Disabilities Monitoring Network. Surveillance Summaries. 2009;58(SS10):1-20. https://www.cdc.gov/mmwr/preview/mmwrhtml/ss5810a1.htm - PubMed
    1. Developmental Disabilities Monitoring Network Surveillance Year 2010 Principal Investigators. Centers for Disease Control and Prevention (CDC) Prevalence of autism spectrum disorder among children aged 8 years - autism and developmental disabilities monitoring network, 11 sites, United States, 2010. MMWR Surveill Summ. 2014 Mar 28;63(2):1–21. https://www.cdc.gov/mmwr/preview/mmwrhtml/ss6302a1.htm - PubMed
    1. Tyler Carl, Schramm Sarah, Karafa Matthew, Tang Anne S, Jain Anil. Electronic Health Record Analysis of the Primary Care of Adults with Intellectual and Other Developmental Disabilities. J Policy Pract Intellect Disabil. 2010 Sep;7(3):204–210. doi: 10.1111/j.1741-1130.2010.00266.x. http://europepmc.org/abstract/MED/26113866 - DOI - PMC - PubMed

Publication types