Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2023 Jan;65(1):e2100349.
doi: 10.1002/bimj.202100349. Epub 2022 Aug 7.

On the relevance of prognostic information for clinical trials: A theoretical quantification

Affiliations
Randomized Controlled Trial

On the relevance of prognostic information for clinical trials: A theoretical quantification

Sandra Siegfried et al. Biom J. 2023 Jan.

Abstract

The question of how individual patient data from cohort studies or historical clinical trials can be leveraged for designing more powerful, or smaller yet equally powerful, clinical trials becomes increasingly important in the era of digitalization. Today, the traditional statistical analyses approaches may seem questionable to practitioners in light of ubiquitous historical prognostic information. Several methodological developments aim at incorporating historical information in the design and analysis of future clinical trials, most importantly Bayesian information borrowing, propensity score methods, stratification, and covariate adjustment. Adjusting the analysis with respect to a prognostic score, which was obtained from some model applied to historical data, received renewed interest from a machine learning perspective, and we study the potential of this approach for randomized clinical trials. In an idealized situation of a normal outcome in a two-arm trial with 1:1 allocation, we derive a simple sample size reduction formula as a function of two criteria characterizing the prognostic score: (1) the coefficient of determination R2 on historical data and (2) the correlation ρ between the estimated and the true unknown prognostic scores. While maintaining the same power, the original total sample size n planned for the unadjusted analysis reduces to ( 1 - R 2 ρ 2 ) × n $(1 - R^2 \rho ^2) \times n$ in an adjusted analysis. Robustness in less ideal situations was assessed empirically. We conclude that there is potential for substantially more powerful or smaller trials, but only when prognostic scores can be accurately estimated.

Keywords: clinical trials; covariate adjustment; machine learning; prognostic covariates; sample size reduction.

PubMed Disclaimer

Conflict of interest statement

The authors have declared no conflict of interest.

Figures

FIGURE 1
FIGURE 1
Simulated fraction of residual variances in a model with prognostic score as defined in (5). The fractions are shown for the normal linear model regressing on the treatment effect (E(Yz); left), and the model additionally adjusting for the prognostic score estimate (E(Yz,s(x)); right) for various values of R2=π2/σ2 and different sample sizes n= 50, 100, and 10,000. The light gray line depicts the theoretical fraction 1R2ρ^2, with the precision of the random forest ρ^ estimated from the data. The variance reduction predicted by the “design factor” (Borm et al., 2007) is shown as dashed dark gray line
FIGURE 2
FIGURE 2
Characteristics of the employed prognostic model s(X). The precision of the random forest ρ^ estimated from the data are shown along different values of R2=π2/σ2 for different historical sample sizes n= 50, 100, and 10,000
FIGURE 3
FIGURE 3
Simulated distribution of the treatment effect estimate. The treatment effect estimates β^ from the normal linear model regressing on the treatment effect (E(Yz); left), and the model additionally adjusting for the prognostic score estimate (E(Yz,s(x)); right) are shown for various values of R2=π2/σ2 and different sample sizes n= 50, 100, and 10,000. The true treatment effect β=0.12 is indicated by the horizontal line

Similar articles

Cited by

References

    1. Anonymous (2022). Draft qualification opinion for prognostic covariate adjustment (PROCOVA TM ) . EMA/SA/0000059571, European Medicines Agency. https://www.ema.europa.eu
    1. Arbogast, P. G. , & Ray, W. A. (2009). Use of disease risk scores in pharmacoepidemiologic studies. Statistical Methods in Medical Research, 18(1), 67–80. 10.1177/0962280208092347 - DOI - PubMed
    1. Arbogast, P. G. , & Ray, W. A. (2011). Performance of disease risk scores, propensity scores, and traditional multivariable outcome regression in the presence of multiple confounders. American Journal of Epidemiology, 174(5), 613–620. 10.1093/aje/kwr143 - DOI - PubMed
    1. Athey, S. , & Imbens, G. (2017). The econometrics of randomized experiments. In Banerjee A. V., and Duflo E., (Eds.), Handbook of economic field experiments, (Vol. 1, pp. 73–140). North‐Holland. 10.1016/bs.hefe.2016.10.003 - DOI
    1. Borm, G. F. , Fransen, J. , & Lemmens, W. A. (2007). A simple sample size formula for analysis of covariance in randomized clinical trials. Journal of Clinical Epidemiology, 60(12), 1234–1238. 10.1016/j.jclinepi.2007.02.006 - DOI - PubMed

Publication types