Automated ICD coding via unsupervised knowledge integration (UNITE)

Abstract

Objective: Accurate coding is critical for medical billing and electronic medical record (EMR)-based research. Recent research has been focused on developing supervised methods to automatically assign International Classification of Diseases (ICD) codes from clinical notes. However, supervised approaches rely on ICD code data stored in the hospital EMR system and is subject to bias rising from the practice and coding behavior. Consequently, portability of trained supervised algorithms to external EMR systems may suffer.

Method: We developed an unsupervised knowledge integration (UNITE) algorithm to automatically assign ICD codes for a specific disease by analyzing clinical narrative notes via semantic relevance assessment. The algorithm was validated using coded ICD data for 6 diseases from Partners HealthCare (PHS) Biobank and Medical Information Mart for Intensive Care (MIMIC-III). We compared the performance of UNITE against penalized logistic regression (LR), topic modeling, and neural network models within each EMR system. We additionally evaluated the portability of UNITE by training at PHS Biobank and validating at MIMIC-III, and vice versa.

Results: UNITE achieved an averaged AUC of 0.91 at PHS and 0.92 at MIMIC over 6 diseases, comparable to LR and MLP. It had substantially better performance than topic models. In regards to portability, the performance of UNITE was consistent across different EMR systems, superior to LR, topic models and neural network models.

Conclusion: UNITE accurately assigns ICD code in EMR without requiring human labor, and has major advantages over commonly used machine learning approaches. In addition, the UNITE attained stable performance and high portability across EMRs in different institutions.

Keywords: Automated ICD assignment; Electronic medical records; Knowledge integration; Portability; Semantic embedding; Unsupervised learning.

Figure 1.

Workflow of UNITE in predicting the presence of ICD for one specific disease. NLP: natural language processing; NER: named entity recognition; CUI: concept unique identifier. β is the regression coefficient which serves as the weights for importance of the CUIs.

Figure 2.

Relative CUI importance for RA, CAD, UC, MS, LC, CD using a word cloud representation with the magnitude of the font size proportional to the CUI importance.

Figure 3.

AUC for penalized logistic regression (LR), penalized logistic regression with UNITE (UNITE-LR), penalized logistic regression with the mean CUI (mCUI-LR), penalized logistic regression with the TFIDF (TFIDF-LR), multiple layer perceptron (MLP), multiple layer perceptron with UNITE features (UNITE-MLP), multiple layer perceptron with the mean CUI (mCUI-MLP), multiple layer perceptron with TFIDF (TFIDF-MLP), topic modeling based on LDA with 2 topics fit with VEM (LDA_VEM) and Gibbs (LDA_Gibbs), CTM with 2 topics fit with VEM (CTM_VEM) and UNITE. Each row is a different disease, columns show performance for methods trained and tested on either the same hospital system (first two panels), or trained and tested in different hospital systems (last two panels).

Figure 4.

F-score for penalized logistic regression (LR), penalized logistic regression with UNITE (UNITE-LR), penalized logistic regression with the mean CUI (mCUI-LR), penalized logistic regression with the TFIDF (TFIDF-LR), multiple layer perceptron (MLP), multiple layer perceptron with UNITE features (UNITE-MLP), multiple layer perceptron with the mean CUI (mCUI-MLP), multiple layer perceptron with TFIDF (TFIDF-MLP), topic modeling based on LDA with 2 topics fit with VEM (LDA_VEM) and Gibbs (LDA_Gibbs), CTM with 2 topics fit with VEM (CTM_VEM) and UNITE.