Enhancing cardiac hemodynamic and pulsatility in heart failure via deep reinforcement learning: An in-silico and in-vitro validation study of percutaneous ventricular assist devices

Abstract

Background and objective: Percutaneous ventricular assist devices (pVADs) are critical for bridging heart failure (HF) patients to recovery or transplantation, yet existing control strategies-constant speed control and preprogrammed pulsatile control-lack adaptability to dynamic physiological variations, leading to reduced pulsatility and hemodynamic mismatch. This study proposes a deep reinforcement learning (DRL)-based adaptive control framework to optimize pVAD performance. The goal is to restore physiological pulsatile hemodynamics while autonomously adjusting to different HF conditions, heart rate fluctuations, and intra-cycle ejection phase variability.

Methods: Following a dual-validation pathway designed to bridge simulation with physical testing, a cardiovascular-pVAD in-silico model was developed and its fidelity confirmed against an in-vitro pulsatile mock circulatory loop. This validated platform was then used to design and test the DRL controller. A modified Deep Deterministic Policy Gradient (DDPG) algorithm with embedded LSTM layers was designed to capture temporal characteristics in aortic pressure (AOP) and aortic flow(AF) waveforms. The reward function integrated hemodynamic recoverability, pulsatile waveform similarity, and control stability and safety penalty.

Results: Comparative simulations and experiments demonstrated the DRL controller's superiority over conventional strategies. Under the moderate HF condition, DRL controller achieved near-physiological AOP (DTW-AOP: 1.17 vs. 16.42 for constant speed control; 2.72 for preprogrammed pulsatile control) and AF (DTW-AF: 21.23 vs. 71.74/48.96), with pulsatility indices (PI: 1.69 vs. 1.05/1.54) and pulse pressures (PP: 34.42 mmHg vs. 3.20/24.90 mmHg) closely matching healthy reference. The framework exhibited robust adaptability to heart rate shifts (75→120 bpm) and ejection phase delays (0.1 s), maintaining stability despite sensor noise and physiological perturbations.

Conclusions: This DRL controller enables real-time synchronization with native cardiac cycles and generalization across pathologies, paving the way for precision pVAD support and future clinical translation.

Keywords: Adaptive control; Deep reinforcement learning; Heart failure; Hemodynamics; Pulsatility; Ventricular assist devices.