Grad-CAM Guided U-Net for MRI-based Pseudo-CT Synthesis

Abstract

In this paper, we address the task of image-to-image translation from MRI to CT domain. We propose a 2D U-Net-based deep learning approach for pseudo-CT synthesis that incorporates an additional Grad-CAM guided attention mechanism for superior image translation of bone regions. The suggested architecture consists of image-to-image translation and image classification modules. We first train our classifier to distinguish between MR and CT images. After that, we utilize it in combination with the Grad-CAM technique to provide additional guidance to our image-to-image translation network. We generate CT-class-specific localization maps for both CT and pseudo-CT images and then compare them. Thus, we force the image-to-image translation network to focus on relevant attributes of the CT class, such as bone structures, while learning to synthesize pseudo-CTs. The performance of the proposed approach is evaluated on the publicly available RIRE data set. Since MR and CT images in this data set are not correctly aligned with each other, we also briefly describe the applied image registration procedure. The experimental results are compared to the baseline U-Net model and demonstrate both qualitative and quantitative improvements, whereas significant performance gain is achieved for bone regions. Clinical Relevance- MRI-based pseudo-CT synthesis is essential for attenuation correction of PET in combined PET/MRI systems and plays a vital role in MRI-only radiotherapy planning. Accurate pseudo-CTs can prevent patients from harmful and unnecessary radiation exposure.