Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2019 Feb 28;14(2):e0212778.
doi: 10.1371/journal.pone.0212778. eCollection 2019.

Natural language processing and machine learning algorithm to identify brain MRI reports with acute ischemic stroke

Chulho Kim  1   2   3 , Vivienne Zhu  2   3 , Jihad Obeid  2   3 , Leslie Lenert  2   3   4
Affiliations
Comparative Study

Natural language processing and machine learning algorithm to identify brain MRI reports with acute ischemic stroke

Chulho Kim et al. PLoS One. .

Abstract

Background and purpose: This project assessed performance of natural language processing (NLP) and machine learning (ML) algorithms for classification of brain MRI radiology reports into acute ischemic stroke (AIS) and non-AIS phenotypes.

Materials and methods: All brain MRI reports from a single academic institution over a two year period were randomly divided into 2 groups for ML: training (70%) and testing (30%). Using "quanteda" NLP package, all text data were parsed into tokens to create the data frequency matrix. Ten-fold cross-validation was applied for bias correction of the training set. Labeling for AIS was performed manually, identifying clinical notes. We applied binary logistic regression, naïve Bayesian classification, single decision tree, and support vector machine for the binary classifiers, and we assessed performance of the algorithms by F1-measure. We also assessed how n-grams or term frequency-inverse document frequency weighting affected the performance of the algorithms.

Results: Of all 3,204 brain MRI documents, 432 (14.3%) were labeled as AIS. AIS documents were longer in character length than those of non-AIS (median [interquartile range]; 551 [377-681] vs. 309 [164-396]). Of all ML algorithms, single decision tree had the highest F1-measure (93.2) and accuracy (98.0%). Adding bigrams to the ML model improved F1-mesaure of naïve Bayesian classification, but not in others, and term frequency-inverse document frequency weighting to data frequency matrix did not show any additional performance improvements.

Conclusions: Supervised ML based NLP algorithms are useful for automatic classification of brain MRI reports for identification of AIS patients. Single decision tree was the best classifier to identify brain MRI reports with AIS.

PubMed Disclaimer

Conflict of interest statement

Drs. Kim, Zhu and Obeid have no competing interests. Dr. Lenert is a member of the Board of Directors of the ATCC. This does not alter the authors' adherence to PLOS ONE policies on sharing data and materials.

Figures

Fig 1
Fig 1. Preprocessing flow chart of “quanteda” natural language processing package.
Fig 2
Fig 2. Difference of the text character lengths between AIS and non-AIS reports.
AIS, acute ischemic stroke.
Fig 3
Fig 3. Result of keyness plot analysis of AIS and non-AIS reports.
AIS, acute ischemic stroke.
Fig 4
Fig 4. Comparison of ML and NLP algorithms for classifying the brain MRI reports.
ML, machine learning; NLP, natural language processing, BLR, binary logistic regression; NBC, naïve Bayesian classification; SDT, single decision tree; SVM, support vector machine; TFIDF, term frequency-inverse document frequency.
See this image and copyright information in PMC

References

    1. GBD 2015 DALYs and HALE Collaborators. Global, regional, and national disability-adjusted life-years (DALYs) for 315 diseases and injuries and healthy life expectancy (HALE), 1990–2015: a systematic analysis for the global burden of disease study 2015. Lancet. 2016;388:1603–1658. 10.1016/S0140-6736(16)31460-X - DOI - PMC - PubMed
    1. GBD 2015 Mortality and Causes of Death Collaborators. Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: a systematic analysis for the global burden of disease study 2015. Lancet. 2016;388:1459–1544. 10.1016/S0140-6736(16)31012-1 - DOI - PMC - PubMed
    1. Murray CJ, Ezzati M, Flaxman AD, Lim S, Lozano R, Michaud C, et al. GBD 2010: a multi-investigator collaboration for global comparative descriptive epidemiology. Lancet. 2012;380:2055–2058. - PubMed
    1. Krishnamurthi RV, Barker-Collo S, Parag V, Parmar P, Witt E, Jones A, et al. Stroke incidence by major pathological type and ischemic subtypes in the Auckland regional community stroke studies: changes between 2002 and 2011. Stroke. 2018;49:3–10. 10.1161/STROKEAHA.117.019358 - DOI - PubMed
    1. Koton S, Schneider AL, Rosamond WD, Shahar E, Sang Y, Gottesman RF, et al. Stroke incidence and mortality trends in US communities, 1987 to 2011. JAMA. 2014;312:259–268. 10.1001/jama.2014.7692 - DOI - PubMed

Publication types