Color Rendering in Medical Extended-Reality Applications

Abstract

Cross-platform development of medical applications in extended-reality (XR) head-mounted displays (HMDs) often relies on game engines with rendering capabilities currently not standardized in the context of medical visualizations. Many aspects of the visualization pipeline including the characterization of color have yet to be consistently defined across rendering models and platforms. We examined the transfer of color properties from digital objects, through the rendering and image processing steps, to the RGB values sent to the display device. Five rendering pipeline configurations within the Unity engine were evaluated using 24 digital color patches. In the second experiment, the same configurations were evaluated with a tissue slide sample image. Measurements of the change in color associated with each configuration were characterized using the CIE 1976 color difference ([Formula: see text]). We found that the distribution of [Formula: see text] for the first experiment ranges from zero, as in the case using an Unlit Shader, to 25.97, as in the case using default configurations. The default Unity configuration consistently returned the highest [Formula: see text] across all 24 colors and also the largest range of color differences. In the second experiment, [Formula: see text]E ranged from 7.49 to 34.18. The Unlit configuration resulted in the highest [Formula: see text] in three of four selected pixels in the tissue sample image. Changes in color image properties associated with texture import settings were then evaluated in a third experiment using the TG18-QC test pattern. Differences in pixel values were found in all nine of the investigated texture import settings. The findings provide an initial characterization of color transfer and a basis for future work on standardization, consistency, and optimization of color in medical XR applications.

Keywords: Color transfer; Graphics rendering; Image quality; Medical imaging; Mixed reality; Virtual reality.

Fig. 1

RGB input and output for five digital material, digital lighting, and digital camera configurations within the Unity engine in the rendering of cross-platform applications for selected scenes: (a) a digital pathology image [8], (b) a digital chest radiograph, and (c) a full-field digital mammogram [9]

Fig. 2

Setup of the Unity scene for all three experiments

Fig. 3

Four pixels varying in RGB values are selected from the ground truth of a tissue sample slide obtained from multispectral imaging [8] as input RGB data

Fig. 4

Input and rendered output for 24 color patches of ColorChecker across five configurations described in Table 2. Reference white (255, 255, 255) can be seen on the upper left corner of each configuration

Fig. 5

Mean $\mathrm{\Delta }\text{E}$ for color patch outputs and selected pixels across the five configurations for experiments 1(a) and 2(b)

Fig. 6

(a) TG18-QC test pattern render output with increasing incremental smoothness to the right. Top row is metallic 0, bottom row at metallic 1. (b) Rendered output next to the input image file. (c) Detailed view of input versus output

Fig. 7

Pixel by pixel differences between the render output image and original TG18-QC test pattern

Fig. 8

Histograms of the number of pixels with the difference from 0-255 (0-25 shown here) between original 1024x1024 TIFF image TG18-QC pattern and the output 1024x1024 PNG images using 9 different import settings rendered using Unlit Shader