A multi-sequences MRI deep framework study applied to glioma classfication

Abstract

Glioma is one of the most important central nervous system tumors, ranked 15th in the most common cancer for men and women. Magnetic Resonance Imaging (MRI) represents a common tool for medical experts to the diagnosis of glioma. A set of multi-sequences from an MRI is selected according to the severity of the pathology. Our proposed approach aims moreto create a computer-aided system that is capable of helping morethe expert diagnose the brain gliomas. moreWe propose a supervised learning regime based on a convolutional neural network based framework and transfer learning techniques. Our research morefocuses on the performance of different pre-trained deep learning models with respect to different MRI sequences. We highlight the best combinations of such model-MRI sequence couple for our specific task of classifying healthy brain against brain with glioma. moreWe also propose to visually analyze the extracted deep features for studying the existing relation of the MRI sequences and models. This interpretability analysis gives some hints for medical expert to understand the diagnosis made by the models. Our study is based on the well-known BraTS datasets including multi-sequence images and expert diagnosis.

Keywords: Deep leaning model; Glioma classification; MRI; Model interpretability; Multi-sequences.

Fig. 1

Pipeline of the current study

Fig. 2

Examples of an identical transverse section of a patient, who suffers from a high grade glioma in T1 sequence (a), T1ce sequence (b), T2 sequence (c) and T2-FLAIR (d)

Fig. 3

Scheme of an implemented model

Fig. 4

Global weights analysis (a) and weights analysis by MRI sequences (b)

Fig. 5

Frequency usage of each sequence according to accuracy of the combination (a) 2 sequences out of 16, (b) 3 out of 16 and (c) 4 out of 16. All combination were sort decreasingly by accuracy. In y axis, the number of occurrence of these modalities found and in x axis the number of the combination for the four different CNNs: Resnet50, DenseNet, InceptionV3, and VGG19 and the four different sequences : T1, T2, T2FLAIR, and T1ce

Fig. 6

Frequency usage of each sequence according to accuracy of the combination 3 out of 20. All combination were sort decreasingly by accuracy. In y axis, the number of occurrence of these modalities found and in x axis the number of the combination for the five different CNNs: Resnet50, DenseNet, InceptionV3, EfficientNet-B6, and VGG19 and the four different sequences : T1, T2, T2FLAIR, and T1ce for the database BraTS 2020

Fig. 7

Transverse slice of a patient suffering from a high grade glioma using different sequences: T1ce (a), T2 (b), T2 FLAIR (c), and combination of the 3 sequences as an RGB image (d)

Fig. 8

Accuracy and loss of selected CNN models : ResNet50 and DenseNet. For each subfigures, in x axis the number of epochs for the training, and in y axis the value of the accuracy of the train and validation set

Fig. 9

Accuracy with respect to combinations of sequences for UNET3D. In x axis the different combination of sequences. In y axis the accuracy

Fig. 10

Confusion matrix for UNET3D for the combination of T2 FLAIR, T1 and T2 sequences. HGG is the label for the presence of a glioma, NG is for healthy brain

Fig. 11

Frequency usage of each sequence according to accuracy of the combination (a) 3 sequences out of 20, (b) Frequency usage of each neural network according to accuracy of the combination (a) 3 sequences out of 20. In y axis, the number of occurrence of these modalities found and in x axis the number of the combination

Fig. 12

Deep feature maps of Epsilon-LRP on a trichromatic (T2-FLAIR, T2, T1ce) image (row 1), Deep Taylor on T2-FLAIR sequence (row 2), Guided Backpropagation on T1 sequence (row 3), using DenseNet (column 1), InceptionV3 (column 2), Resnet50 (column 3) and VGG19 (column 4)

Fig. 13

Input image (column 1), deep feature maps from ResNet50 using Epsilon-LRP (column2), Deep Taylor (column 3) and Guided Backpropagation (column 4), of a high grade glioma of RGB (row 1), T1 (row 2), T2 (row 3), T1ce (row 4) and T2 FLAIR (row 5)