Explainable hierarchical clustering for patient subtyping and risk prediction

Abstract

We present a pipeline in which machine learning techniques are used to automatically identify and evaluate subtypes of hospital patients admitted between 2017 and 2021 in a large UK teaching hospital. Patient clusters are determined using routinely collected hospital data, such as those used in the UK's National Early Warning Score 2 (NEWS2). An iterative, hierarchical clustering process was used to identify the minimum set of relevant features for cluster separation. With the use of state-of-the-art explainability techniques, the identified subtypes are interpreted and assigned clinical meaning, illustrating their robustness. In parallel, clinicians assessed intracluster similarities and intercluster differences of the identified patient subtypes within the context of their clinical knowledge. For each cluster, outcome prediction models were trained and their forecasting ability was illustrated against the NEWS2 of the unclustered patient cohort. These preliminary results suggest that subtype models can outperform the established NEWS2 method, providing improved prediction of patient deterioration. By considering both the computational outputs and clinician-based explanations in patient subtyping, we aim to highlight the mutual benefit of combining machine learning techniques with clinical expertise.

Keywords: Hierarchical clustering; clinical evaluation; early warning score; explainability; mortality prediction; patient subtypes.

Figure 1.

Pipeline overview, from data set import to generation of explainable clusters and clinical outcome predictions. The blue box denotes the iterative clustering process.

Figure 2.

Patients mapped onto the two-dimensional embedding space after dimensionality reduction and clustering. Clusters inside black boxes depict the subclustering results. Subclustering was not performed for clusters 4 and 6 as both contained less than 1000 patients. Unclusterable patients are shown in dark blue, often at the edges of clusters.

Figure 3.

NEWS, vitals, age, gender, ICD-10 code count, and length of stay for individual clusters from clustering the entire population. The mean value of each cluster is compared to the mean or median value depending on the feature (black line) of the whole population.

Figure 4.

NEWS, vitals, age, gender, ICD-10 code count, and length of stay show the subclustering results for cluster 0. The mean value of each cluster is compared to the mean/median value (black line) of the parent cluster.

Figure 5.

NEWS, vitals, age, gender, ICD-10 code count, and length of stay show the subclustering results for cluster 1. The mean value of each cluster is compared to the mean/median value (black line) of the parent cluster.

Figure 6.

Heatmap of primary ICD-10 codes of full-population clustering and collated by top-level grouping. For display purposes, only ICD-10 codes with ⩾ 2% incidence for at least one cluster are displayed. Since only a subset of ICD-10 codes are visualized, each row does not add up to 100. MSK: musculoskeletal.

Figure 7.

Heatmap of primary ICD-10 codes of the subclusters of cluster 0 as recorded by clinicians at the time of patient admission and collated by top-level grouping. For display purposes, only ICD-10 codes with ⩾ 2% incidence for at least one cluster are displayed. Since only a subset of ICD-10 codes are visualized, each row does not add up to 100. MSK: musculoskeletal.

Figure 8.

Heatmap of primary ICD-10 codes of different subclusters as recorded by clinicians at the time of patient admission and collated by top-level grouping. For display purposes, only ICD-10 codes with ⩾ 2% incidence for at least one cluster are displayed. Since only a subset of ICD-10 codes are visualized, each row does not add up to 100. MSK: musculoskeletal.

Figure 9.

Surrogate explanations for the contribution of each vital in determining the assignment of patients into each cluster from clustering of the entire population. SATS: hemoglobin saturation with oxygen.

Figure 10.

Surrogate explanations for the contribution of each vital in determining the assignment of patients into each subcluster of cluster 0. SATS: hemoglobin saturation with oxygen.

Figure 11.

Surrogate explanations for the contribution of each vital in determining the assignment of patients into each subcluster of cluster 1. SATS: hemoglobin saturation with oxygen.

Figure 12.

Predictive performance for classification models compared against the existing NEWS2 risk scoring system for the two predicted outcomes: in-hospital mortality and admission to higher care (general ICU, cardiac ICU, and critical care unit). “All” refers to the entire unclustered patient cohort. Mortality was not predicted for cluster 0 since only one positive case occurred: (a) mortality ROC, (b) mortality PRC, (c) admission to higher care ROC, and (d) admission to higher care PRC. Figures in brackets are the area under the curve. ROC: receiver operating characteristic curves; PRC: precision recall curve.