CardioAI: A Multimodal AI-based System to Support Symptom Monitoring and Risk Prediction of Cancer Treatment-Induced Cardiotoxicity

Abstract

Despite recent advances in cancer treatments that prolong patients' lives, treatment-induced cardiotoxicity (i.e., the various heart damages caused by cancer treatments) emerges as one major side effect. The clinical decision-making process of cardiotoxicity is challenging, as early symptoms may happen in non-clinical settings and are too subtle to be noticed until life-threatening events occur at a later stage; clinicians already have a high workload focusing on the cancer treatment, no additional effort to spare on the cardiotoxicity side effect. Our project starts with a participatory design study with 11 clinicians to understand their decision-making practices and their feedback on an initial design of an AI-based decision-support system. Based on their feedback, we then propose a multimodal AI system, CardioAI, that can integrate wearables data and voice assistant data to model a patient's cardiotoxicity risk to support clinicians' decision-making. We conclude our paper with a small-scale heuristic evaluation with four experts and the discussion of future design considerations.

Keywords: Cancer treatment-induced cardiotoxicity; Human-AI collaboration; Large Language Models; Multimodal AI system.

Figure 1:

CardioAI: a multimodal AI system to support clinicians for remote monitoring and risk detection of cancer patients’ cardiotoxicity risk. The UI has five modules: (A) Patient Information; (B) AI-generated Daily Summary; (C) Wearable Sensor Data; (D) AI-generated and Explainable Risk Score; (E) Conversation Log.

Figure 2:

Our participatory design session. A participant is suggesting design revisions on the initial UI.

Figure 3:

Examples of low-fidelity visualization options for heart rate data used to elicit clinician feedback.

Figure 4:

AI-based Cardiotoxicity Risk Score Prediction Framework.

Figure 5:

System Architecture of CardioAI. It integrates a wearable and a smart speaker to continuously collect physiological data and patient-reported symptoms. Data is processed by three key LLM-powered backend components: a conversation module, a summarization module, and a risk prediction module. The processed data, including key health summaries and cardiotoxicity risk scores with explainability, is then visualized on a clinician-facing dashboard.