Characterizing the clinical profile and prevalence of people with diabetes attended in the hospital setting by using unstructured healthcare data and natural language processing: the Diabetic@ study

Abstract

Aims: This study aimed to evaluate the potential of unstructured electronic health records (EHRs) data, analyzed using natural language processing (NLP) and machine learning (ML), to describe the prevalence and clinical spectrum of diabetes mellitus (DM) in hospitals.

Methods: A multicenter, retrospective study was conducted using EHRs from eight Spanish hospitals (2013-2018). Unstructured data were extracted using EHRead® (NLP and ML) and SNOMED CT. Individuals with type 1 or 2 DM (T1DM/T2DM) were identified, and a semi-supervised ML classifier was developed for unregistered types (UrDM). DM prevalence and related complications were analyzed in the final subpopulations (sT1DM/sT2DM).

Results: From 56,181,954 EHRs of 2,582,778 individuals, 638,730 were identified with DM: 75.4% with UrDM, 21.3% with T2DM, and 3.3% with T1DM. The ML model reclassified 93.5% as T2DM and 6.5% as T1DM. Over 50% of relevant variables like anthropometrics, laboratory values and treatments were missing. The prevalence of sT1DM/sT2DM was 2.6%/38.4%. Major comorbidities included hypertension, dyslipidemia, chronic kidney disease (CKD), ischemic heart disease, and chronic heart failure (CHF). CKD and CHF were the most frequent complications for sT1DM/sT2DM at 60 months.

Conclusions: NLP and ML for profiling DM using EHRs unstructured data are helpful, but additional data and better EHR documentation are crucial.

Keywords: Diabetes Comorbidities; Diabetes Complications; Diabetes Mellitus (DM); Machine Learning (ML); Natural Language Processing (NLP); Unstructured Data.