A Novel Detection and Classification Framework for Diagnosing of Cerebral Microbleeds Using Transformer and Language

Abstract

The detection of Cerebral Microbleeds (CMBs) is crucial for diagnosing cerebral small vessel disease. However, due to the small size and subtle appearance of CMBs in susceptibility-weighted imaging (SWI), manual detection is both time-consuming and labor-intensive. Meanwhile, the presence of similar-looking features in SWI images demands significant expertise from clinicians, further complicating this process. Recently, there has been a significant advancement in automated detection of CMBs using a Convolutional Neural Network (CNN) structure, aiming at enhancing diagnostic efficiency for neurologists. However, existing methods still show discrepancies when compared to the actual clinical diagnostic process. To bridge this gap, we introduce a novel multimodal detection and classification framework for CMBs' diagnosis, termed MM-UniCMBs. This framework includes a light-weight detection model and a multi-modal classification network. Specifically, we proposed a new CMBs detection network, CMBs-YOLO, designed to capture the salient features of CMBs in SWI images. Additionally, we design an innovative language-vision classification network, CMBsFormer (CF), which integrates patient textual descriptions-such as gender, age, and medical history-with image data. The MM-UniCMBs framework is designed to closely align with the diagnostic workflow of clinicians, offering greater interpretability and flexibility compared to existing methods. Extensive experimental results show that MM-UniCMBs achieves a sensitivity of 94% in CMBs' classification and can process a patient's data within 5 s.

Keywords: cerebral microbleeds; convolutional neural network; detection and classification; language–vision; multimodal.

Figure A1

The detection results of CMB-YOLO (X6).

Figure A2

The classification results of CF (B) with text input.

Figure 1

The architecture of MM-UniCMBs.

Figure 2

The structure of CMBs-YOLO.

Figure 3

Feature responses of CMBs in MRI.

Figure 4

The overview of the CMBs-MHA module.

Figure 5

The relationship between the predicted boxes and ground truth boxes.

Figure 6

The architecture of CMBsFormer.

Figure 7

The evaluation of various models across different metrics in the CMBs-Private testing dataset.

Figure 8

Comparing the trade-off between speed and performance among detection models in the CMBs-Private testing dataset.

Figure 9

Comparing the trade-off between parameters and performance among different detection models in the CMBs-Private testing dataset.

Figure 10

Visual results of detection models in CMBs-Private testing dataset.

Figure 11

Comparing the trade-off between speed and performance in the CMBs-Private testing dataset.

Figure 12

Comparing the trade-off between parameters and performance in the CMBs-Private testing dataset.

Figure 13

Comparing the trade-off between speed and performance among different models’ combinations in the CMBs-Private testing dataset.

Figure 14

The impact of CMBs-MHA on detection models at various scales in the CMBs-Private testing dataset.

Figure 15

The loss function variation curve during training with the CMBs-Private training dataset.

Figure 16

Performance variations of different detection models under the 768-resolution training mode.

Figure 17

Impact of resolution changes on model prediction scores.

Figure 18

Performance variations of different classification models in dynamic mode.

Figure 19

Different classification network structures based on CLIP.

Figure 20

Performance comparison of different strategies under CLIP.