Exploring time series of hyperspectral images for cold water coral stress response analysis

Abstract

Hyperspectral imaging (HSI) is a promising technology for environmental monitoring with a lot of undeveloped potential due to the high dimensionality and complexity of the data. If temporal effects are studied, such as in a monitoring context, the analysis becomes more challenging as time is added to the dimensions of space (image coordinates) and wavelengths. We conducted a series of laboratory experiments to investigate the impact of different stressor exposure patterns on the spectrum of the cold water coral Desmophyllum pertusum. 65 coral samples were divided into 12 groups, each group being exposed to different types and levels of particles. Hyperspectral images of the coral samples were collected at four time points from prior to exposure to 6 weeks after exposure. To investigate the relationships between the corals' spectral signatures and controlled experimental parameters, a new software tool for interactive visual exploration was developed and applied, the HypIX (Hyperspectral Image eXplorer) web tool. HypIX combines principles from exploratory data analysis, information visualization and machine learning-based dimension reduction. This combination enables users to select regions of interest (ROI) in all dimensions (2D space, time point and spectrum) for a flexible integrated inspection. We propose two HypIX workflows to find relationships in time series of hyperspectral datasets, namely morphology-based filtering workflow and embedded driven response analysis workflow. With these HypIX workflows three users identified different temporal and spatial patterns in the spectrum of corals exposed to different particle stressor conditions. Corals exposed to particles tended to have a larger change rate than control corals, which was evident as a shifted spectrum. The responses, however, were not uniform for coral samples undergoing the same exposure treatments, indicating individual tolerance levels. We also observed a good inter-observer agreement between the three HyPIX users, indicating that the proposed workflow can be applied to obtain reproducible HSI analysis results.

Fig 1. Overview of the Hypix workflow.

a) Corals are kept in glass aquariums. b) Different aquariums are exposed with either barite, bentonite, drill cutting or kept as is for the control experiment. c) The exposure happens at a timepoint T0.5. Before exposure (T0), directly after exposure (T0.5) and after 2 (T1) and 6 weeks (T2) of recovery hyperspectral images are taken. d) The images are preprocessed. e) The outline of the corals is annotated in Biigle 2.0 and the individual corals are cut out for further processing. f) Data mining and machine learning methods are applied and saved to a file for g) Visualization and analysis of the results in the Hypix system. For more details on each step please have a look at the respective subsection in the Method section.

Fig 2. Screenshot of the HypIX tool: In the top frame (a) the experimental conditions, concentration levels, dimension reduction algorithms and individual coral samples (0, …, 4 or 5) can be chosen by the user.

On the right (b) a pseudo image of each HSI of all four time steps is shown in the image display, in this case the mean spectral response value (see Methods for details). On the right side in frame (b) the user can chose to apply image normalization or use pseudocolor to change the images visualization. In frame (c), spectral signatures from all four time steps are shown in the spectra display. This window initially shows the agglomerated spectral signatures from all four data sets color encoded (T0: blue, T0.5: yellow, T1: red, T2: green. (d) The large frame on the left (d) shows the dimension reduction results for the four HSI data sets in the embedding display using again the same color code for the four time points.

Fig 3. Local changes revealed with the interactive subselection of the hyperspectral viewer, otherwise shadowed by global analysis.

Shown is the bentonite sample with 100mg ⋅ L⁻¹ concentration using a UMAP with cosine metric and coral 3. The normalization of spectra was activated and the aggregation option was set to mean.

Fig 4. Subjective rating results of the changes from initial time point T0 to final time point T2 of the bentonite, barite and DC experiment after six weeks.

All values represent the average over all subjective ratings. Ratings range from 1 (no change) to 5 (strong change). Analogues to the color scale of the Hyperspectral Viewer, yellow is T0.5 (directly after exposure), red is T1 (two weeks after exposure) and green is T2 (six weeks after exposure). The titles of the subgraphs are the concentrations in mg ⋅ L⁻¹ of the exposure depicted as the row title.

Fig 5. Pseudocolor images of each timepoint.

We can clearly spot differences between time point T0 and the other time points. a) start of experiment (T0); b) immediately after exposure (T0.5); c) after 2 weeks of exposure (T1); d) after 6 weeks of exposure (T2).

Fig 6. Screenshot of the spectral view of Hypix.

Example for a red-shifted spectrum due to the exposure with a barite concentration of 30mg ⋅ L⁻¹ for coral 0. The blue part of the light spectrum is lowered while the red part is elevated at T0.5, T1 and T2 compared to before the exposure at T0.