Frequency-aware optical coherence tomography image super-resolution via conditional generative adversarial neural network

Abstract

Optical coherence tomography (OCT) has stimulated a wide range of medical image-based diagnosis and treatment in fields such as cardiology and ophthalmology. Such applications can be further facilitated by deep learning-based super-resolution technology, which improves the capability of resolving morphological structures. However, existing deep learning-based method only focuses on spatial distribution and disregards frequency fidelity in image reconstruction, leading to a frequency bias. To overcome this limitation, we propose a frequency-aware super-resolution framework that integrates three critical frequency-based modules (i.e., frequency transformation, frequency skip connection, and frequency alignment) and frequency-based loss function into a conditional generative adversarial network (cGAN). We conducted a large-scale quantitative study from an existing coronary OCT dataset to demonstrate the superiority of our proposed framework over existing deep learning frameworks. In addition, we confirmed the generalizability of our framework by applying it to fish corneal images and rat retinal images, demonstrating its capability to super-resolve morphological details in eye imaging.

Fig. 1.

Frequency domain gaps between the HR and the SR OCT images generated by four CNN implementations (MSRN, RDN, RDU, RCAN). The spectrum is generated by performing Fourier transform on the B-scan image. The high-frequency components of images are generated by performing an inverse Fourier transform on the high-frequency parts of the spectrum. Compared to the HR image, SR images generated by existing CNN algorithms are biased to a limited spectrum region towards low-frequency. Using ours CNN algorithm with frequency awareness (ours), the spectrum and high-frequency components of the SR OCT image are closer to that of the original image. The scale bar represents 500μm.

Fig. 2.

The design of the proposed frequency-aware framework for OCT image super-resolution. The proposed model utilizes wavelet transformation, frequency skip connection, and high-frequency alignment to facilitate frequency information for super-resolving OCT images.

Fig. 3.

Frequency analysis of the SR OCT images generated from LR OCT data acquired using factors of X2, X3, and X4. Compared to existing methods, our frequency-aware model is capable of super-resolving OCT images with less spectrum bias, which is confirmed by frequency analysis.

Fig. 4.

Generating SR OCT images of stent structure from LR image acquired using a factor of X4. The corresponding histology image is attached. Our model resolves the boundary between the stent and tissue with better details due to its frequency-awareness design. ROIs are marked by red rectangles. The EPI score of the ROIs is calculated and displayed. The scale bar represents 100μm.

Fig. 5.

Generating SR OCT image of suspicious macrophage regions from LR image acquired using a factor of X4. The amplitude of intensities of HR and SR images is attached. Our model resolves the accumulations of macrophages without blurring effects. ROIs containing the macrophage accumulations are marked by —-red rectangles. ROIs of edge regions are marked by orange rectangles. The EPI score of the ROIs is calculated and displayed. The scale bar represents 100μm.

Fig. 6.

Generating SR OCT images of anterior segments in fish eyes from LR images acquired using a factor of X3. ROIs are marked by red rectangles. The textures are highlighted by the dashed cycles. The scale bar represents 500μm.

Fig. 7.

Generating SR OCT images of posterior segments in rat eyes from LR images acquired using a factor of X3. ROIs are marked by red rectangles. The textures are highlighted by the white arrows. The scale bar represents 500μm.