A Tutorial on Time-Dependent Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example

Abstract

In an introductory tutorial, we illustrated building cohort state-transition models (cSTMs) in R, where the state transition probabilities were constant over time. However, in practice, many cSTMs require transitions, rewards, or both to vary over time (time dependent). This tutorial illustrates adding 2 types of time dependence using a previously published cost-effectiveness analysis of multiple strategies as an example. The first is simulation-time dependence, which allows for the transition probabilities to vary as a function of time as measured since the start of the simulation (e.g., varying probability of death as the cohort ages). The second is state-residence time dependence, allowing for history by tracking the time spent in any particular health state using tunnel states. We use these time-dependent cSTMs to conduct cost-effectiveness and probabilistic sensitivity analyses. We also obtain various epidemiological outcomes of interest from the outputs generated from the cSTM, such as survival probability and disease prevalence, often used for model calibration and validation. We present the mathematical notation first, followed by the R code to execute the calculations. The full R code is provided in a public code repository for broader implementation.

Keywords: R software; cohort state-transition models; cost-effectiveness analysis; markov models; time-dependent; tutorial.

Figure 1:

A 3-dimensional representation of the transition probability array of the Sick-Sicker model with simulation-time dependence.

Figure 2:

Cohort trace of the age-dependent cohort state- transition models under the strategy of standard of care with no treatment.

Figure 3:

State-transition diagram of the Sick-Sicker model with tunnel states expanding the Sick state $\left(S{1}_1,S{1}_2,\dots ,S{1}_{n_{\text{tunnels}}}\right)$. The circles represent the health states, and the arrows represent the possible transition probabilities. The labels next to the arrows represent the variable names for these transitions.

Figure 4:

The 3-dimensional transition probability array of the Sick-Sicker model expanded to account for simulation-time and state-residence time dependence using τ tunnel states for S1.

Figure 5:

Cost-effectiveness efficient frontier of all four strategies for the simulation-time-dependent Sick-Sicker model.

Figure 6:

Figures generated from the probabilistic sensitivity analysis (PSA) output. A) Cost-effectiveness scatter plot. B) Cost-effectiveness acceptability curves (CEACs) and frontier (CEAF). C) Expected loss curves (ELCs). D) Expected value of perfect information (EVPI).

Figure 7:

Epidemiological outcomes generated from the probabilistic sensitivity analysis (PSA) output for the simulation-time dependent cSTM. A) Survival curve. B) Posterior density of life expectancy. C) Prevalence of sick states over time. The shaded area in A and C shows the 95% posterior model-predictive interval of the outcomes and colored lines shows the posterior model-predicted mean based on 1,000 simulations.