End-to-end prognostication in pancreatic cancer by multimodal deep learning: a retrospective, multicenter study

Abstract

Objectives: Pancreatic cancer treatment plans involving surgery and/or chemotherapy are highly dependent on disease stage. However, current staging systems are ineffective and poorly correlated with survival outcomes. We investigate how artificial intelligence (AI) can enhance prognostic accuracy in pancreatic cancer by integrating multiple data sources.

Materials and methods: Patients with histopathology and/or radiology/follow-up confirmed pancreatic ductal adenocarcinoma (PDAC) from a Dutch center (2004-2023) were included in the development cohort. Two additional PDAC cohorts from a Dutch and Spanish center were used for external validation. Prognostic models including clinical variables, contrast-enhanced CT images, and a combination of both were developed to predict high-risk short-term survival. All models were trained using five-fold cross-validation and assessed by the area under the time-dependent receiver operating characteristic curve (AUC).

Results: The models were developed on 401 patients (203 females, 198 males, median survival (OS) = 347 days, IQR: 171-585), with 98 (24.4%) short-term survivors (OS < 230 days) and 303 (75.6%) long-term survivors. The external validation cohorts included 361 patients (165 females, 138 males, median OS = 404 days, IQR: 173-736), with 110 (30.5%) short-term survivors and 251 (69.5%) longer survivors. The best AUC for predicting short vs. long-term survival was achieved with the multi-modal model (AUC = 0.637 (95% CI: 0.500-0.774)) in the internal validation set. External validation showed AUCs of 0.571 (95% CI: 0.453-0.689) and 0.675 (95% CI: 0.593-0.757).

Conclusion: Multimodal AI can predict long vs. short-term survival in PDAC patients, showing potential as a prognostic tool in clinical decision-making.

Key points: Question Prognostic tools for pancreatic ductal adenocarcinoma (PDAC) remain limited, with TNM staging offering suboptimal accuracy in predicting patient survival outcomes. Findings The multimodal AI model demonstrated improved prognostic performance over TNM and unimodal models for predicting short- and long-term survival in PDAC patients. Clinical relevance Multimodal AI provides enhanced prognostic accuracy compared to current staging systems, potentially improving clinical decision-making and personalized management strategies for PDAC patients.

Keywords: Artificial intelligence; Pancreas; Pancreatic ductal carcinoma; Prognosis; Radiology.

Fig. 1

Flowchart of selected PDAC patients. There were 974 selected PDAC patients from CENTER 1 and CENTER 2. After exclusion, 753 patients remained. An additional external cohort, CENTER 3, was added, including 109 PDAC patients

Fig. 2

Schematic overview of the multimodal survival prediction pipeline. Overall survival (OS) scores float between 0 and 1, indicating the likelihood of a patient being a short-term survivor

Fig. 3

Kaplan–Meier curves for identified short (red) and long (blue) survivors for the unimodal clinical, imaging, and multimodal AI models for CENTER 1: validation cohort, CENTER 2, and CENTER 3

Fig. 4

Patient examples of the identified short- and long survivors in the CENTER 1: validation set, CENTER 2, and CENTER 3

Fig. 5

Kaplan–Meier curves for all identified TNM stages in the CENTER 1: development cohort, CENTER 1: validation cohort, CENTER 2, and CENTER 3