Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection

Abstract

Brain surgery is one of the most common and effective treatments for brain tumour. However, neurosurgeons face the challenge of determining the boundaries of the tumour to achieve maximum resection, while avoiding damage to normal tissue that may cause neurological sequelae to patients. Hyperspectral (HS) imaging (HSI) has shown remarkable results as a diagnostic tool for tumour detection in different medical applications. In this work, we demonstrate, with a robust k-fold cross-validation approach, that HSI combined with the proposed processing framework is a promising intraoperative tool for in-vivo identification and delineation of brain tumours, including both primary (high-grade and low-grade) and secondary tumours. Analysis of the in-vivo brain database, consisting of 61 HS images from 34 different patients, achieve a highest median macro F1-Score result of 70.2 ± 7.9% on the test set using both spectral and spatial information. Here, we provide a benchmark based on machine learning for further developments in the field of in-vivo brain tumour detection and delineation using hyperspectral imaging to be used as a real-time decision support tool during neurosurgical workflows.

Fig. 1. Proposed intraoperative HSI approach for surgical assistance.

a HSI concept explanation. b Synthetic RGB images of a surface-layer tumour (left) and a deep-layer tumour (right). Tumour area is surrounded in yellow by a clinical expert. c HS data acquisition and labelling procedure during surgery. In the ground-truth map, red represents tumour tissue (TT) labelled pixels, green normal tissue (NT) pixels, blue blood vessel (BV) pixels, and black background (BG) pixels. Meanwhile, white represents non-labelled pixels. VIS Visual; VNIR Visual and Near Infrared; SWIR Short-Wave Infrared; MIR Mid-Infrared; SRGB Synthetic Red-Green-Blue.

Fig. 2. Spectral characterization of different brain tissue and tumour types.

Mean (solid lines) and standard deviation (std) (dashed lines) of the entire labelled dataset after applying a basic pre-processing (calibration, extreme band noise removal, and noise filtering) and separated by classes, including the corresponding p-value (magenta dots) computed for each spectral channel using the paired two-sided Wilcoxon Rank Sum test at 5% of significance level between the two compared classes. a TT vs. NT class. b TT vs. BV class. c Primary vs. secondary tumours. d HG vs. LG primary tumours. e G1 vs. G2 primary tumours. f G3 vs. G4 primary tumours.

Fig. 3. Spectral characterization of TT, NT, and BV classes and their relationship with HbO₂ and deoxyHb.

Mean absorbance values of the entire labelled dataset separated by classes (solid lines) after applying a basic pre-processing (calibration, extreme band noise removal, and noise filtering) and molar extinction spectra (dashed lines) of HbO₂ and deoxyHb. a TT class between 440 and 650 nm. b TT class between 650 and 910 nm. c NT class between 440 and 650 nm d NT class between 650 and 910 nm. e BV class between 440 and 650 nm. f BV class between 650 and 910 nm.

Fig. 4. Spectral classification results of brain tissue.

a Boxplots of the macro F1-Score results of the validation set for each training data reduction and each classifier, including the 5 folds using the optimal hyperparameters in each classifier. In the plot, the centre line, the box limits and the whiskers represent the median, the upper and lower quartiles and the 1.5× interquartile range, respectively. Two medians are significantly different at the 5% significance level if their intervals (shaded colour areas) do not overlap. b Average OA, sensitivity, and specificity results of the validation set from the 5 folds using the data reduction of 1000 pixels per class (error bars represent the standard deviation). c Examples of SRGB images, GT maps and supervised classification maps generated using the EBEAE and DNN algorithms with the optimal hyperparameters from different tumour types of the validation set. Approximate tumour areas were surrounded in yellow lines on the SRGB image by the operating surgeon according to the intraoperative neuronavigation and the definitive pathological diagnosis of the resected tissue. Rubber ring markers were employed in some cases (e.g., Op8C1) to indicate the area where the biopsies for pathology were resected. Opx Operation number x; Cy Capture number y.

Fig. 5. Quantitative and qualitative results at the different stages of the proposed framework in the validation set.

a Boxplots of the macro F1-Score of the validation set using the eight different classifiers at the three different stages. In the plot, the centre line, the box limits and the whiskers represent the median, the upper and lower quartiles and the 1.5× interquartile range, respectively. Two medians are significantly different at the 5% significance level if their intervals (shaded colour areas) do not overlap. b Average OA, sensitivity, and specificity results of the validation set from the 5 folds using the Spatial/Spectral approach (error bars represent the standard deviation). c Example of the output maps at the different stages of the proposed framework from Op42C2 of the validation set (based on the DNN as supervised algorithm using the optimal hyperparameters).

Fig. 6. Quantitative results at the different stages of the proposed framework and qualitative TMD classification maps in the test set.

a Boxplots of the macro F1-Score of the test set using the eight different classifiers at the three different stages. In the plot, the centre line, the box limits and the whiskers represent the median, the upper and lower quartiles and the 1.5× interquartile range, respectively. Two medians are significantly different at the 5% significance level if their intervals (shaded colour areas) do not overlap. b Average OA, sensitivity, and specificity results of the test set from the 5 folds using the Spatial/Spectral approach (error bars represent the standard deviation). c Examples of SRGB images, GT maps and TMD maps from different tumour types (based on the DNN as supervised algorithm using the optimal hyperparameters).

Fig. 7. Post-hoc interpretability using LIME approach.

Graphical representation of the ten most relevant features identified for the classification models by the RF, KNN-E, KNN-C and DNN algorithms using the training set of the first fold in each class. a Average spectral signature of the NT class. b Average spectral signature of the TT class. c Average spectral signature of the BV class. d Average spectral signature of the BG class. In the plot, the size of the markers represents the level of importance computed by LIME (higher size is related to higher importance). Positive coefficients are represented at the top of the chart while negative coefficients are shown at the bottom. RF is only evaluated with positive predictor importance values.

Fig. 8. Data partition and proposed processing framework.

a Patient/image flow scheme of this work and data partition. n Number of HS images, m number of patients. b Proposed processing framework to generate the density maps for intraoperative pathology-assisted surgery.