Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Feb 9;5(3):229-234.
doi: 10.1093/ehjdh/ztae008. eCollection 2024 May.

Using natural language processing for automated classification of disease and to identify misclassified ICD codes in cardiac disease

Maarten Falter  1   2   3 , Dries Godderis  4 , Martijn Scherrenberg  1   2   5 , Sevda Ece Kizilkilic  1   2   6 , Linqi Xu  1   2 , Marc Mertens  7 , Jan Jansen  7 , Pascal Legroux  7 , Hanne Kindermans  1 , Peter Sinnaeve  3 , Frank Neven  4 , Paul Dendale  1   2
Affiliations

Using natural language processing for automated classification of disease and to identify misclassified ICD codes in cardiac disease

Maarten Falter et al. Eur Heart J Digit Health. .

Abstract

Aims: ICD codes are used for classification of hospitalizations. The codes are used for administrative, financial, and research purposes. It is known, however, that errors occur. Natural language processing (NLP) offers promising solutions for optimizing the process. To investigate methods for automatic classification of disease in unstructured medical records using NLP and to compare these to conventional ICD coding.

Methods and results: Two datasets were used: the open-source Medical Information Mart for Intensive Care (MIMIC)-III dataset (n = 55.177) and a dataset from a hospital in Belgium (n = 12.706). Automated searches using NLP algorithms were performed for the diagnoses 'atrial fibrillation (AF)' and 'heart failure (HF)'. Four methods were used: rule-based search, logistic regression, term frequency-inverse document frequency (TF-IDF), Extreme Gradient Boosting (XGBoost), and Bio-Bidirectional Encoder Representations from Transformers (BioBERT). All algorithms were developed on the MIMIC-III dataset. The best performing algorithm was then deployed on the Belgian dataset. After preprocessing a total of 1438 reports was retained in the Belgian dataset. XGBoost on TF-IDF matrix resulted in an accuracy of 0.94 and 0.92 for AF and HF, respectively. There were 211 mismatches between algorithm and ICD codes. One hundred and three were due to a difference in data availability or differing definitions. In the remaining 108 mismatches, 70% were due to incorrect labelling by the algorithm and 30% were due to erroneous ICD coding (2% of total hospitalizations).

Conclusion: A newly developed NLP algorithm attained a high accuracy for classifying disease in medical records. XGBoost outperformed the deep learning technique BioBERT. NLP algorithms could be used to identify ICD-coding errors and optimize and support the ICD-coding process.

Keywords: Atrial fibrillation; BioBERT; Deep learning; Heart failure; ICD codes; International classification of disease; Machine learning; Natural language processing; XGBoost.

PubMed Disclaimer

Conflict of interest statement

Conflict of interest: None declared

Figures

Graphical Abstract
Graphical Abstract
See this image and copyright information in PMC

References

    1. Hirsch JA, Nicola G, McGinty G, Liu RW, Barr RM, Chittle MD, et al. . ICD-10: history and context. AJNR Am J Neuroradiol 2016;37:596–599. - PMC - PubMed
    1. Burns EM, Rigby E, Mamidanna R, Bottle A, Aylin P, Ziprin P, et al. . Systematic review of discharge coding accuracy. J Public Health (Oxf) 2012;34:138–148. - PMC - PubMed
    1. Bosco-Lévy P, Duret S, Picard F, dos Santos P, Puymirat E, Gilleron V, et al. . Diagnostic accuracy of the international classification of diseases, tenth revision, codes of heart failure in an administrative database. Pharmacoepidemiol Drug Saf 2019;28:194–200. - PubMed
    1. Champagnie SJ. Medicare loses billions to billing errors (September 5, 2019). In: Proceedings of the Ninth International Conference on Engaged Management Scholarship, 2019. doi:10.2139/ssrn.3454108. - DOI
    1. Portuondo JI, Harris AHS, Massarweh NN. Using administrative codes to measure health care quality. JAMA 2022;328:825–826. - PubMed

LinkOut - more resources