Estimating optimal shared-parameter dynamic regimens with application to a multistage depression clinical trial

Abstract

A dynamic treatment regimen consists of decision rules that recommend how to individualize treatment to patients based on available treatment and covariate history. In many scientific domains, these decision rules are shared across stages of intervention. As an illustrative example, we discuss STAR*D, a multistage randomized clinical trial for treating major depression. Estimating these shared decision rules often amounts to estimating parameters indexing the decision rules that are shared across stages. In this article, we propose a novel simultaneous estimation procedure for the shared parameters based on Q-learning. We provide an extensive simulation study to illustrate the merit of the proposed method over simple competitors, in terms of the treatment allocation matching of the procedure with the "oracle" procedure, defined as the one that makes treatment recommendations based on the true parameter values as opposed to their estimates. We also look at bias and mean squared error of the individual parameter-estimates as secondary metrics. Finally, we analyze the STAR*D data using the proposed method.

Keywords: Dynamic treatment regimens; Q-learning; STAR*D; Shared parameters.

Figure 1

A schematic of the treatment assignment algorithm in the STAR*D study. An “R” within a circle denotes randomization.

Figure 2

Convergence patterns of ψ₀, ψ₁, ψ₂, ψ₃ and ψ₄ for the five versions of the Q-shared method (corresponding to five initial values) in the STAR*D study.

Figure 3

Confidence planes for the contrast functions and resulting regions of varying recommended optimal treatments in the (QIDS.start, QIDS.slope) plane, for subjects who have experienced low side effect intensity (0) and were treated with combination therapy (−1) at the previous stage, based on the Q-shared method for estimation and m-out-of-n bootstrap for inference.