Virtual reality for the observation of oncology models (VROOM): immersive analytics for oncology patient cohorts

Abstract

The significant advancement of inexpensive and portable virtual reality (VR) and augmented reality devices has re-energised the research in the immersive analytics field. The immersive environment is different from a traditional 2D display used to analyse 3D data as it provides a unified environment that supports immersion in a 3D scene, gestural interaction, haptic feedback and spatial audio. Genomic data analysis has been used in oncology to understand better the relationship between genetic profile, cancer type, and treatment option. This paper proposes a novel immersive analytics tool for cancer patient cohorts in a virtual reality environment, virtual reality to observe oncology data models. We utilise immersive technologies to analyse the gene expression and clinical data of a cohort of cancer patients. Various machine learning algorithms and visualisation methods have also been deployed in VR to enhance the data interrogation process. This is supported with established 2D visual analytics and graphical methods in bioinformatics, such as scatter plots, descriptive statistical information, linear regression, box plot and heatmap into our visualisation. Our approach allows the clinician to interrogate the information that is familiar and meaningful to them while providing them immersive analytics capabilities to make new discoveries toward personalised medicine.

Figure 1

The Visualisation has four datasets rendered in similarity space based on five different dimensional reduction algorithms. (a) Our user interfaces with the options available for detailed analysis. (b) The four primary datasets categorised into two and three risk factors with Autoencoder, NMF, PCA, tSNE, and UMAP dimensional reduction algorithm.

Figure 2

A close-up of the 3D similarity space of the patient cohort for the OHSC threerisk dataset. Colours are used to indicate the risk level. The cohort can roughly be divided into three regions, a, b and c.

Figure 3

Patient "aml_ohsu_2018_16-01272" is selected from (a) and drag into (b) patient of Interest panel. The panel shows the metadata for the patient, such as age, gender and ethnicity. (c) The tablet also contains the box plot of the patient gene data compared to the whole population. The patient gene expression value is clearly marked as an "x" (yellow) in the box plot. (d) A bar chart of the gene expression value.

Figure 4

Patient-to-Patient Comparison panel. (a) The patient “aml_ohsu_2018_16-01272” is our POI, circled with green, while "aml_ohsu_2018_16-00271" and “aml_ohsu_2018_16-00316” are our POCs, circled with white. (b) POC—"aml_ohsu_2018_16-00271” is used to show the dissimilarity of the gene expression from POI. The scatter plot is used to analyse the similarity between the patient in term of their gene expression. These two patients have a similarity rate of 85%. When regression was carried out on the data, the model does not fit well, with only 0.179 R-square value. (c) In contrast, “aml_ohsu_2018_16-00316″ is very similar to POI at 91.5%. Regression model fit well with R-square value of 0.731.

Figure 5

In the 3D similarity space, our POI is highlighted and linked by straight lines to some other patients as suggested by our AI. (a) To use the Decision Support System., the analyst can push the "Patient Of Interest Menu" button. The list of top 5 combos will be listed on the left side of the screen as shown in (b). The details of the gene description extracted from GeneCards® are displayed on the right. This is one of the Overview and details analysis keys to visual analytics that can be conducted within this similarity space. By selecting different combinations, the neighbours linked by the straight line will change if that neighbour is no longer part of the relationship. This is particularly useful if the gene combination has a special meaning. The selected groups of POCs can be then sent to Patient-to-Group Panel for further analysis. In this case, the selected combination is “KIAA0125”, “SOC2” and “AKR1C3”, which was found using NMF, and the result was a Nash Equilibrium and social optimal solution for game theory.

Figure 6

Patient-to-Group comparison panel. The analyst using the decision support system to trigger the selection of POI and POCs in this panel. The primary patient is our POI as shown in (a). All POCs are highlighted in light blue in (a) and are listed under the group panel in (b).

Figure 7

(a) The Patient-to-Group comparison panel has four visual analytics tools, hierarchical clustering heatmap, box plot, patient table and 3D heatmap. (b) The gene expression of the POI and the POCs are visualised in the heatmap. (c) Patients clinical history in tabular format. (d) The Box plot comparison of POI to POCs. The box plot shows the gene's median and IQR for the POI and the POCS. (e) 3D visualisation of the heatmap with the gene expression value determines the height of the mountain and the valley.

Figure 8

The heatmap is integrated with the hierarchy clustering and dendrogram. Red line is used to indicate the current level of clustering for both gene and sample. (a) The gene-level clustering was almost at the top of the tree. This created two clusters for the gene expression, as seen by the two different colours, in the row headers. (b) The level for gene clustering was lower to 10, which is about the middle of the tree hierarchy. At this stage, there are eight distinct clusters can be found for the gene. A similar study can be done for the sample hierarchy. Heatmap and dendrogram are very popular for gene expression visualisation.

Figure 9

A walkthrough of an end-to-end analysis using the patient details panel and patient-to-group panel for overview and details analysis to unleash the knowledge embedded deep in the data for our POI and POCs. (a) First, the analyst can investigate the gene-of-interest, “CDK6”, expression value for the POI to see if it falls in the IQR range or an outliner. The analyst then can use the POCs suggested by the Decision Support System to investigate why these POCs cohort mostly survived the illness. (b) The analyst can then inspect the gene expression value for the POI compared to the POCs and conduct a hierarchical clustering analysis on the heatmap. Since the grouping was based on “KIAA0125”, “SOC2” and “AKR1C3” gene combination, the analyst can visually compare the differential expression for these three genes for POI and POCs. Since the similarity was done on these three genes, it made sense that those values shown in (b) were close to each other. (c) The medical history for all the patients. In this scenario, the analyst would find the POI and patient "aml_ohsu_2018_15-00371" were dead; and compare the regimen used for these patients. The visual comparison reveals that the regimen used for our POI is not the same as any of the rest of the POCs.

Figure 10

The analyst can stack all the panels anywhere in the immersive space to allow them to conduct a detailed analysis. This is possible in immersive space as there is unlimited space that can be used.

Figure 11

Our immersive visual analytics of genomic data for cancer patients extends Nguyen et al..