A Cross-Modal Mutual Knowledge Distillation Framework for Alzheimer's Disease Diagnosis: Addressing Incomplete Modalities

Abstract

Early detection of Alzheimer's Disease (AD) is crucial for timely interventions and optimizing treatment outcomes. Despite the promise of integrating multimodal neuroimages such as MRI and PET, handling datasets with incomplete modalities remains under-researched. This phenomenon, however, is common in real-world scenarios as not every patient has all modalities due to practical constraints such as cost, access, and safety concerns. We propose a deep learning framework employing cross-modal Mutual Knowledge Distillation (MKD) to model different sub-cohorts of patients based on their available modalities. In MKD, the multimodal model (e.g., MRI and PET) serves as a teacher, while the single-modality model (e.g., MRI only) is the student. Our MKD framework features three components: a Modality-Disentangling Teacher (MDT) model designed through information disentanglement, a student model that learns from classification errors and MDT's knowledge, and the teacher model enhanced via distilling the student's single-modal feature extraction capabilities. Moreover, we show the effectiveness of the proposed method through theoretical analysis and validate its performance with simulation studies. In addition, our method is demonstrated through a case study with Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets, underscoring the potential of artificial intelligence in addressing incomplete multimodal neuroimaging datasets and advancing early AD detection.

Note to practitioners—: This paper was motivated by the challenge of early AD diagnosis, particularly in scenarios when clinicians encounter varied availability of patient imaging data, such as MRI and PET scans, often constrained by cost or accessibility issues. We propose an incomplete multimodal learning framework that produces tailored models for patients with only MRI and patients with both MRI and PET. This approach improves the accuracy and effectiveness of early AD diagnosis, especially when imaging resources are limited, via bi-directional knowledge transfer. We introduced a teacher model that prioritizes extracting common information between different modalities, significantly enhancing the student model's learning process. This paper includes theoretical analysis, simulation study, and real-world case study to illustrate the method's promising potential in early AD detection. However, practitioners should be mindful of the complexities involved in model tuning. Future work will focus on improving model interpretability and expanding its application. This includes developing methods to discover the key brain regions for predictions, enhancing clinical trust, and extending the framework to incorporate a broader range of imaging modalities, demographic information, and clinical data. These advancements aim to provide a more comprehensive view of patient health and improve diagnostic accuracy across various neurodegenerative diseases.

Keywords: Alzheimer’s disease; incomplete multimodal datasets; knowledge distillation; mild cognitive impairment; representation disentanglement.

Fig. 1:

A conceptual depiction of the proposed MKD framework. The teacher, trained with more modalities, achieves higher accuracy, while the student, trained with fewer modalities but more samples, has better single-modal representation extraction capability. Both models enhance each other by mutually exchanging their respective strengths.

Fig. 2:

Categorization of KD methods: (a) Complex-to-simple standard KD within the same modality, (b) Single-to-single cross-modal KD, and (c) Multi-to-single cross-modal KD.

Fig. 3:

Teacher and student model architectures and training process in the MKD framework. MDT/MDT⁺: upper architecture without/with dash lines; MDT-Student: bottom architecture. First, the teacher is designed to disentangle modality-common and -specific information and uses modality-common information for classification (MDT). Second, the student then learns from the teacher through forward KD (MDT-Student). Last, the teacher is subsequently updated by learning the student’s feature extraction capability through reverse KD, and adding modality-specific information (MDT⁺).

Fig. 4:

UMAP visualization of the modality-common $\left({\mathbf{z}}^{ca},{\mathbf{z}}^{cb}\right)$ and modality-specific $\left({\mathbf{z}}^{sa},{\mathbf{z}}^{sb}\right)$ representations in the simulation test set.

Fig. 5:

Hyperparameter tuning results for MKD. (a) ${\alpha }_{sim}$ and ${\alpha }_{diff}$ for MDT-Student, (b) ${\alpha }_{recon}$ for MDT-Student, (c) ${\alpha }_s$ for MDT-Student, and (d) ${\alpha }_t$ for MDT⁺. The highest AUROC value for each experiment is highlighted in dark blue.

Fig. 6:

Comparisons of validation CE loss for MDT models with (blue) and without (red) decoder. Shaded regions represent standard deviations above and below the respective mean values. MDT without decoder overfits, with the CE loss increasing after 30 epochs.