Framework for hyperspectral image processing and quantification for cancer detection during animal tumor surgery

Abstract

Hyperspectral imaging (HSI) is an imaging modality that holds strong potential for rapid cancer detection during image-guided surgery. But the data from HSI often needs to be processed appropriately in order to extract the maximum useful information that differentiates cancer from normal tissue. We proposed a framework for hyperspectral image processing and quantification, which includes a set of steps including image preprocessing, glare removal, feature extraction, and ultimately image classification. The framework has been tested on images from mice with head and neck cancer, using spectra from 450- to 900-nm wavelength. The image analysis computed Fourier coefficients, normalized reflectance, mean, and spectral derivatives for improved accuracy. The experimental results demonstrated the feasibility of the hyperspectral image processing and quantification framework for cancer detection during animal tumor surgery, in a challenging setting where sensitivity can be low due to a modest number of features present, but potential for fast image classification can be high. This HSI approach may have potential application in tumor margin assessment during image-guided surgery, where speed of assessment may be the dominant factor.

Fig. 1

Flowchart of the proposed method.

Fig. 2

Rationale for the proposed glare detection method. (a) Image band at 758 nm. (b) Enlarged image of a selected glare region in the image band in (a). Glare pixels are much brighter than nonglare pixels. (c) Normalized reflectance curve of glare pixels (G1 to G3) and nonglare pixels (NG1 to NG3). Spectral curve of glare pixels varies significantly in many wavelengths. (d) First-order derivative curves corresponding to the spectral curves shown in (c).

Fig. 3

Flowchart for feature selection and classification.

Fig. 4

Glare detection results: (a) Standard deviation (std) image of the first order derivative for a hypercube. (b) Binary glare map generated by the classical Otsu method. (c) Binary glare map generated by the entropy method. (d)–(f) Glare map generated by the proposed method with ratios 0.01, 0.05, and 0.1. (g) Histogram of the std image with blue color and loglogistic fitting curve with red color. The five vertical lines represent the five thresholds generated by the five methods in (b)–(f).

Fig. 5

Visualization of a tumor with green fluorescence protein (GFP). The upper part is the image of the tumor in a mirror. (a) RGB composite image of the hypercube. (b) Preprocessed spectral images at wavelengths 450 nm, 508 nm, 510 nm, 542 nm, 554 nm, 576 nm, 600 nm, and 650 nm. (c) Spectral curve of cancerous and healthy tissue.

Fig. 6

Visualization of a tumor without GFP. (a) RGB composite image of the hypercube. (b) Preprocessed spectral images at wavelengths 450 nm, 508 nm, 510 nm, 542 nm, 554 nm, 576 nm, 600 nm, and 650 nm. (c) Spectral curve of cancerous and healthy tissue.

Fig. 7

Reflectance spectral curve of a tumor with necrosis. (a) RGB composite image of hypercube. The white region looks necrotic, and the other part of the tumor contains many vessels; (b) Histological image of the rectangular tissue region in (a). The upper part is the necrotic tissue without nuclei, and the lower part is the viable cancerous tissue. (c) The average reflectance spectra of the tumor, necrosis, and normal tissue with std. The red solid line represents the average spectra of cancerous tissue, and the blue dotted line represents the average spectra of the normal tissue. The green dashed line represents the average spectra of the necrotic tissue. The error bars are the std at a certain wavelength of the three curves.

Fig. 8

Feature extraction and visualization. (a) RGB composite image of hypercube, mean, std, and sum of the selected tumor and normal tissue ROI. (b)–(e) Average normalized reflectance curve, first derivative, second derivative, and the difference of FCS between normal and tumor tissue in the selected ROI in (a).

Fig. 9

Mutual information between features and class labels. The $x$ axis represents the feature number, and the $y$ axis represents the mutual information.

Fig. 10

Mutual information between individual features. Color bar on the right shows the color map corresponds to the value of mutual information. Higher mutual information indicates more redundancy between features.

Fig. 11

Feature selection and classification.

Fig. 12

Feature ranking. The $x$ axis is the ranking from 1 to 20, and the $y$ axis is the percentage of the selection frequency for different features on each rank. Each bar represents the normalized frequency of different feature types being selected as the $i$th ranked feature.