Deep-learning-based image super-resolution of an end-expandable optical fiber probe for application in esophageal cancer diagnostics

Abstract

Significance: Endoscopic screening for esophageal cancer (EC) may enable early cancer diagnosis and treatment. While optical microendoscopic technology has shown promise in improving specificity, the limited field of view ($<1\mathrm{mm}$) significantly reduces the ability to survey large areas efficiently in EC screening.

Aim: To improve the efficiency of endoscopic screening, we propose a novel concept of end-expandable endoscopic optical fiber probe for larger field of visualization and for the first time evaluate a deep-learning-based image super-resolution (DL-SR) method to overcome the issue of limited sampling capability.

Approach: To demonstrate feasibility of the end-expandable optical fiber probe, DL-SR was applied on simulated low-resolution microendoscopic images to generate super-resolved (SR) ones. Varying the degradation model of image data acquisition, we identified the optimal parameters for optical fiber probe prototyping. The proposed screening method was validated with a human pathology reading study.

Results: For various degradation parameters considered, the DL-SR method demonstrated different levels of improvement of traditional measures of image quality. The endoscopists' interpretations of the SR images were comparable to those performed on the high-resolution ones.

Conclusions: This work suggests avenues for development of DL-SR-enabled sparse image reconstruction to improve high-yield EC screening and similar clinical applications.

Keywords: Barrett’s esophagus; deep-learning-based super-resolution; degradation model; end-expandable optical fiber probe; endomicroscopy; esophageal cancer; microendoscopy.

Fig. 1

Conceptual 3D rendering of a settled optical fiber probe (a) and an end-expanded optical fiber probe (b), respectively. Here, the diameter of a single fiber strand can vary from 4 to $6\mu \mathrm{m}$.

Fig. 2

Schematics of image acquisition by means of (a) a conventional fused endoscopic fiber probe; (b) an end-expandable, unfused endoscopic fiber probe.

Fig. 3

(a) 1D-schematic of the conventional image data acquisition leading to the HR images; (b) the sparse image data acquisition and reconstitution from the compressed data set into the LR images. Red elements represent a fiber strand collecting light illuminated by the tissue surface and green elements are a set of pixels displayed as average value of light intensity in the adjacent fiber. (c) 2D-illustration of the simulated sparse image data; $s$ and $m$ are the length of side of FOV and ROI, respectively; $d_x$ and $d_y$ are the offsets in $x$ and $y$ directions, respectively.

Fig. 4

Illustration of degradation models that incorporated various optical fiber probe parameters including (a) offset, (b) inter-fiber distance, and (c) fiber diameter.

Fig. 5

Example of an original HRME image, the same imaging area as sparse data captured by the proposed end-expandable optical fiber probe, and the LR image restored from the sparse data.

Fig. 6

Schematic of the SRCNN architecture used in the study.

Fig. 7

Examples of original HRME image, simulated LR image and corresponding SR image (top row) generated by the DL-SR method with magnification (bottom row).

Fig. 8

Cross-sectional line profile on selected images allows comparison of profiles of optical intensity associated with stained nuclei on original HR images, simulated LR images, and reconstructed SR images.

Fig. 9

Traditional IQ metrics show degradation of LR and limitation of improvement of SR images with reference to HR images: PSNR and SSIM values of LR (red) and SR (blue). The gray dashed line denotes a SSIM value of 0.95.

Fig. 10

Diagnostic performance of endoscopists’ reading on HR and SR microendoscopy images shown in box and whisker plot. The cross and horizontal line in the box represents mean and median values of the endoscopists’ response. The bottom and the top end from the whiskers are the minimum and maximum values, respectively.

Fig. 11

(a) Sensitivity and specificity plot of individual endoscopist diagnosis on HR and SR microendoscopic images with high and low confidence, respectively. (b) Confusion matrix of diagnostic performance of readers on HR and SR images with low and high confidence, respectively. TP, true positive; FP, false positive; FN, false negative; and TN, true negative.