Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Nov 30;15(1):45769.
doi: 10.1038/s41598-025-28636-9.

A hybrid CNN-ViT framework with cross-attention fusion and data augmentation for robust brain tumor classification

Ganesh Jayaraman  1 , S Meganathan  1 , S Sheik Mohideen Shah  1 , M Anuradha  1 , Ranjeeth Kumar Sundararajan  1 , R Rajakumar  2
Affiliations

A hybrid CNN-ViT framework with cross-attention fusion and data augmentation for robust brain tumor classification

Ganesh Jayaraman et al. Sci Rep. .

Abstract

Brain tumor classification from MRI scans is a challenging task that requires accurate and timely detection increase patient survival rates. Conventional machine learning methods with hand-crafted features often fail to handle different sizes, forms, and textures of tumors. In this study evaluates, standard transfer learning models (AlexNet, MobileNetV2, InceptionV3, ResNet50, VGG16, VGG19) and conventional classifier models such as Decision Tree, Naïve Bayes, LDA were evaluated for multiclass brain tumor classification. The Vision Transformer (ViT) which leverages global context modeling achieved accuracy of 87.34%. To further improve performance, a hybrid CNN–ViT framework named CAFNet with data augmentation and a Cross-Attention Fusion mechanism achieving a test accuracy of 96.41% on a multiclass MRI dataset. The results show that CAFNet significantly outperforms conventional machine learning, deep learning and transfer learning models for robust brain tumor classification.

Keywords: Brain tumor classification; Convolutional neural networks (CNN); Cross-attention fusion; Transfer learning; Vision transformer model.

PubMed Disclaimer

Conflict of interest statement

Declarations. Competing interests: The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Proposed architecture.
Fig. 2
Fig. 2
Training dataset class distribution.
Fig. 3
Fig. 3
Test dataset class distribution.
Fig. 4
Fig. 4
Brain tumor dataset class.
Fig. 5
Fig. 5
Distribution of image widths.
Fig. 6
Fig. 6
Distribution of image heights.
Fig. 7
Fig. 7
Scatter plot of image widths vs heights.
Fig. 8
Fig. 8
Distribution of pixel intensities distribution.
Fig. 9
Fig. 9
Standard ML classifiers performance analysis.
Fig. 10
Fig. 10
Performance analysis of DT, NB and LDA.
Fig. 11
Fig. 11
5-Fold cross validation accuracy.
Fig. 12
Fig. 12
CNN model summary.
Fig. 13
Fig. 13
CNN model accuracy and loss.
Fig. 14
Fig. 14
TL models train and validation accuracy.
Fig. 15
Fig. 15
TL models train and validation loss.
Fig. 16
Fig. 16
TL models training, testing and validation accuracy.
Fig. 17
Fig. 17
Performance analysis for TL Models.
Fig. 18
Fig. 18
ViT prediction model.
Fig. 19
Fig. 19
Train and validation accuracy for ViT model.
Fig. 20
Fig. 20
Train and validation loss for ViT model.
Fig. 21
Fig. 21
Confusion matrix for ViT model.
Fig. 22
Fig. 22
ViT model train, test and validation accuracy.
Fig. 23
Fig. 23
Cross-attention fusion (CAF).
Algorithm
Algorithm
CAFNet (Input Image).
Fig. 24
Fig. 24
CAFNet training and validation accuracy.
Fig. 25
Fig. 25
CAFNet training, validation and testing accuracy.
Fig. 26
Fig. 26
Confusion matrix for CAFNet.
Fig. 27
Fig. 27
Overall Performance Analysis.
Fig. 28
Fig. 28
Ablation study performance analysis.
See this image and copyright information in PMC

References

    1. Kumar, R., Verma, A. & Verma, O. P. Brain tumor segmentation using MRI: A review. IEEE Access.7, 170–187 (2019).
    1. Alqudah, A. M., Alquraan, H., Qasmieh, I. A., Alqudah, A. & Al-Sharu, A. Brain tumor classification using deep learning Technique—A comparison between Cropped, Uncropped, and segmented lesion images with different sizes. Int. J. Adv. Trends Comput. Sci. Eng.9 (1), 111–118 (2020).
    1. Mahmud, T., Sarker, S. & Hossain, M. Deep Learning-Based brain tumor detection: A comprehensive review. IEEE Access.11, 43305–43323 (2023).
    1. Dhakshnamurthy, S., Kumaravel, N. & Manickavasagam, R. Brain Tumor Classification Using Deep Learning Models—A Transfer Learning Approach, Mater. Today Proc., 61, 317–323, (2022).
    1. Kaur, P. & Mahajan, P. Detection of brain tumors using a transfer learning-based optimized ResNet152 model in MR images. Comput. Biol. Med.188, 109790 (2025). - DOI - PubMed

LinkOut - more resources