Bridging pharmacology and neural networks: A deep dive into neural ordinary differential equations

Abstract

The advent of machine learning has led to innovative approaches in dealing with clinical data. Among these, Neural Ordinary Differential Equations (Neural ODEs), hybrid models merging mechanistic with deep learning models have shown promise in accurately modeling continuous dynamical systems. Although initial applications of Neural ODEs in the field of model-informed drug development and clinical pharmacology are becoming evident, applying these models to actual clinical trial datasets-characterized by sparse and irregularly timed measurements-poses several challenges. Traditional models often have limitations with sparse data, highlighting the urgent need to address this issue, potentially through the use of assumptions. This review examines the fundamentals of Neural ODEs, their ability to handle sparse and irregular data, and their applications in model-informed drug development.

FIGURE 1

Neural latent ODE illustration for PK time course prediction for each individual patient. The input consists of including, TFDS, the time in hours between each dose; TIME, the time in hours since the start of the treatment; AMT, the dosing amount in milligrams; CYCL, the current dosing cycle number and finally the PK_Cycle1 as the first cycle of observation. The final hidden state summarizes key information learned. Gaussian sampling derives a latent variable from this state, offering insights into individual patient behavior for precise clinical outcome predictions. Dosing details and the first 20 PK values are added in the Neural ODE decoder, enhancing model accuracy.

FIGURE 2

Depiction of a gated recurrent unit (GRU) in our RNN encoder for irregular data. The GRU assigns weights to covariates. Accurate predictions adjust these weights using stochastic gradient descent, reducing the loss function. High loss may lead to certain covariates getting reduced weights or being deemed irrelevant. This RNN is receiving as input a covariate vector X and the hidden state function at the visit t‐k. This architecture consists of a reset gate (Pink) to discard non‐essential data, an update gate (Green), and a candidate hidden state gate (Purple) to retain pivotal covariates for predicting future clinical outcomes. Different mathematical steps are computed within GRU architecture where some activation functions are represented like tangent hyperbolic function ($\mathit{\text{Tanh}}$) and a random activation function sigma ($\sigma$).

FIGURE 3

Outlined Steps: 1. RNN encoder processes sequences of covariates for all patients, calculating a hidden state between consecutive timesteps. 2. The final hidden function captures crucial information from the patient population, identifying key covariates at a population level. 3. Gaussian sampling introduces variability, creating patient‐specific scenarios to refine individual predictions, generating initial conditions for the latent variable $z_t$., 4. These initial conditions help determine individual latent trajectories for each patient, enabling patient‐level clinical outcome predictions.

FIGURE 4

The Gaussian sampling of the final hidden state sets the initial condition $z_{t0}$ for each patient's latent variable $z_t$., The ODE solver determines individual latent trajectories for patient‐level clinical outcome predictions. These outcomes are refitted over blue dots and extrapolated across yellow dots for multiple visits per patient.,