Optimal treatment recommendations for diabetes patients using the Markov decision process along with the South Korean electronic health records

Abstract

The extensive utilization of electronic health records (EHRs) and the growth of enormous open biomedical datasets has readied the area for applications of computational and machine learning techniques to reveal fundamental patterns. This study's goal is to develop a medical treatment recommendation system using Korean EHRs along with the Markov decision process (MDP). The sharing of EHRs by the National Health Insurance Sharing Service (NHISS) of Korea has made it possible to analyze Koreans' medical data which include treatments, prescriptions, and medical check-up. After considering the merits and effectiveness of such data, we analyzed patients' medical information and recommended optimal pharmaceutical prescriptions for diabetes, which is known to be the most burdensome disease for Koreans. We also proposed an MDP-based treatment recommendation system for diabetic patients to help doctors when prescribing diabetes medications. To build the model, we used the 11-year Korean NHISS database. To overcome the challenge of designing an MDP model, we carefully designed the states, actions, reward functions, and transition probability matrices, which were chosen to balance the tradeoffs between reality and the curse of dimensionality issues.

Figure 1

Diabetes medication prescription percentages in the dataset. This figure shows the diabetes medications prescription by frequency in our dataset. We classified the medication by number of medications prescribed (mono, dual, triple-therapy). The poll shows the percentage of each medications.

Figure 2

Process of data selection and number of patients included in each step. This figure shows the process of data cleansing for experiment.

Figure 3

MDP recommendation actions on each state for male patients. This figure shows the recommended treatment actions according to each component of the states for male patients. (a) Is the recommended actions on complication states. (b) Is the recommended actions on risk states. (c) Is the recommended actions on period state and (d) is the recommended actions on FPG states. Grey shaded sqaure is for mono therapy recommendations. Open square is for dual therapy recommendations. And striped square is for triple therapy recommendations.

Figure 4

MDP recommendation actions on each state for female patients. This figure shows the recommended treatment actions according to each component of the states for female patients. (a) Is the recommended actions on complication states. (b) Is the recommended actions on risk states. (c) Is the recommended actions on period states and (d) is the recommended actions on FPG states. Grey shaded sqaure is for mono therapy recommendations. Open square is for dual therapy recommendations. And striped square is for triple therapy recommendations.

Figure 5

A comparison of the diabetic complication occurrence period between MDP recommendation followers and other prescription users. This figure shows the average period from the time diabetes was diagnosed to the occurrence of complications, and compared the MDP follower groups and other group. Grey shaded sqaure is for MDP follower group and Open square is for other prescription taker group.