Randomized test-treatment studies with an outlook on adaptive designs

doi:10.1186/s12874-021-01293-y

. 2021 Jun 1;21(1):110.

doi: 10.1186/s12874-021-01293-y.

Randomized test-treatment studies with an outlook on adaptive designs

Amra Hot¹, Patrick M Bossuyt², Oke Gerke^{3

4}, Simone Wahl⁵, Werner Vach^#^{6

7}, Antonia Zapf^#⁸

Affiliations

¹ Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, Hamburg, 20246, Germany. a.hot@uke.de.
² Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands.
³ Department of Nuclear Medicine, Odense University Hospital, J.B. Winsløws Vej 4, Odense C, 5000, Denmark.
⁴ Department of Clinical Research, University of Southern Denmark, Winsløwparken 19, Odense C, 5000, Denmark.
⁵ Roche Diagnostics GmbH, Nonnenwald 2, Penzberg, 82377, Germany.
⁶ Basel Academy for Quality and Research in Medicine, Steinenring 6, Basel, 4051, Switzerland.
⁷ Department of Environmental Science, University of Basel, Spalenring 145, Basel, 4055, Switzerland.
⁸ Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, Hamburg, 20246, Germany.

^# Contributed equally.

PMID: 34074263
PMCID: PMC8167391
DOI: 10.1186/s12874-021-01293-y

Randomized test-treatment studies with an outlook on adaptive designs

Amra Hot et al. BMC Med Res Methodol. 2021.

. 2021 Jun 1;21(1):110.

doi: 10.1186/s12874-021-01293-y.

Authors

Amra Hot¹, Patrick M Bossuyt², Oke Gerke^{3

4}, Simone Wahl⁵, Werner Vach^#^{6

7}, Antonia Zapf^#⁸

Affiliations

¹ Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, Hamburg, 20246, Germany. a.hot@uke.de.
² Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands.
³ Department of Nuclear Medicine, Odense University Hospital, J.B. Winsløws Vej 4, Odense C, 5000, Denmark.
⁴ Department of Clinical Research, University of Southern Denmark, Winsløwparken 19, Odense C, 5000, Denmark.
⁵ Roche Diagnostics GmbH, Nonnenwald 2, Penzberg, 82377, Germany.
⁶ Basel Academy for Quality and Research in Medicine, Steinenring 6, Basel, 4051, Switzerland.
⁷ Department of Environmental Science, University of Basel, Spalenring 145, Basel, 4055, Switzerland.
⁸ Institute of Medical Biometry and Epidemiology, University Medical Center Hamburg-Eppendorf, Martinistraße 52, Hamburg, 20246, Germany.

^# Contributed equally.

PMID: 34074263
PMCID: PMC8167391
DOI: 10.1186/s12874-021-01293-y

Abstract

Background: Diagnostic accuracy studies aim to examine the diagnostic accuracy of a new experimental test, but do not address the actual merit of the resulting diagnostic information to a patient in clinical practice. In order to assess the impact of diagnostic information on subsequent treatment strategies regarding patient-relevant outcomes, randomized test-treatment studies were introduced. Various designs for randomized test-treatment studies, including an evaluation of biomarkers as part of randomized biomarker-guided treatment studies, are suggested in the literature, but the nomenclature is not consistent.

Methods: The aim was to provide a clear description of the different study designs within a pre-specified framework, considering their underlying assumptions, advantages as well as limitations and derivation of effect sizes required for sample size calculations. Furthermore, an outlook on adaptive designs within randomized test-treatment studies is given.

Results: The need to integrate adaptive design procedures in randomized test-treatment studies is apparent. The derivation of effect sizes induces that sample size calculation will always be based on rather vague assumptions resulting in over- or underpowered study results. Therefore, it might be advantageous to conduct a sample size re-estimation based on a nuisance parameter during the ongoing trial.

Conclusions: Due to their increased complexity, compared to common treatment trials, the implementation of randomized test-treatment studies poses practical challenges including a huge uncertainty regarding study parameters like the expected outcome in specific subgroups or disease prevalence which might affect the sample size calculation. Since research on adaptive designs within randomized test-treatment studies is limited so far, further research is recommended.

Keywords: Accuracy; Adaptive design; Diagnostic research; Patient-relevant outcome; RCT; Sample size; Test-treatment.

PubMed Disclaimer

Conflict of interest statement

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

**Fig. 1**
A schematic representation of a classical test-treatment RCT [9]

**Fig. 2**
A schematic representation of a randomized diagnostic study with restricting randomization to discordant pairs [9, 25]

**Fig. 3**
A schematic representation of a study design with random disclosure principle [9]

**Fig. 4**
A schematic representation of a study design with random disclosure of one diagnostic test [9]

See this image and copyright information in PMC

Cited by

Evaluation of point-of-care multiplex polymerase chain reaction in guiding antibiotic treatment of patients acutely admitted with suspected community-acquired pneumonia in Denmark: A multicentre randomised controlled trial.
Cartuliares MB, Rosenvinge FS, Mogensen CB, Skovsted TA, Andersen SL, Østergaard C, Pedersen AK, Skjøt-Arkil H. Cartuliares MB, et al. PLoS Med. 2023 Nov 28;20(11):e1004314. doi: 10.1371/journal.pmed.1004314. eCollection 2023 Nov. PLoS Med. 2023. PMID: 38015833 Free PMC article. Clinical Trial.
Towards Recommendations for Cost-Effectiveness Analysis of Predictive, Prognostic, and Serial Biomarker Tests in Oncology.
Kramer A, van Schaik LF, van den Broek D, Meijer GA, Gutierrez Ibarluzea I, Galnares Cordero L, Fijneman RJA, Ligtenberg MJL, Schuuring E, van Harten WH, Coupé VMH, Retèl VP; COIN Consortium. Kramer A, et al. Pharmacoeconomics. 2025 May;43(5):483-497. doi: 10.1007/s40273-025-01470-7. Epub 2025 Feb 8. Pharmacoeconomics. 2025. PMID: 39920559 Free PMC article.
A diagnostic phase III/IV seamless design to investigate the diagnostic accuracy and clinical effectiveness using the example of HEDOS and HEDOS II.
Pepić A, Stark M, Friede T, Kopp-Schneider A, Calderazzo S, Reichert M, Wolf M, Wirth U, Schopf S, Zapf A. Pepić A, et al. Stat Methods Med Res. 2024 Mar;33(3):433-448. doi: 10.1177/09622802241227951. Epub 2024 Feb 7. Stat Methods Med Res. 2024. PMID: 38327081 Free PMC article. Clinical Trial.
2-[18F]FDG-PET/CT in Cancer of Unknown Primary Tumor-A Retrospective Register-Based Cohort Study.
Rimer H, Jensen MS, Dahlsgaard-Wallenius SE, Eckhoff L, Thye-Rønn P, Kristiansen C, Hildebrandt MG, Gerke O. Rimer H, et al. J Imaging. 2023 Aug 31;9(9):178. doi: 10.3390/jimaging9090178. J Imaging. 2023. PMID: 37754942 Free PMC article.
A unified framework for diagnostic test development and evaluation during outbreaks of emerging infections.
Chaturvedi M, Köster D, Bossuyt PM, Gerke O, Jurke A, Kretzschmar ME, Lütgehetmann M, Mikolajczyk R, Reitsma JB, Schneiderhan-Marra N, Siebert U, Stekly C, Ehret C, Rübsamen N, Karch A, Zapf A. Chaturvedi M, et al. Commun Med (Lond). 2024 Dec 10;4(1):263. doi: 10.1038/s43856-024-00691-9. Commun Med (Lond). 2024. PMID: 39658579 Free PMC article. Review.

See all "Cited by" articles

References

1. Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, Williams JW, Kunz R, Craig J, Montori VM, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336(7653):1106–10. doi: 10.1136/bmj.39500.677199.AE. - DOI - PMC - PubMed
1. di Ruffano LF, Hyde CJ, McCaffery KJ, Bossuyt PMM, Deeks JJ. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ. 2012;344:686. doi: 10.1136/bmj.e686. - DOI - PubMed
1. Lijmer JG, Leeflang M, Bossuyt PMM. Proposals for a phased evaluation of medical tests. Med Decis Making. 2009;29(5):13–21. doi: 10.1177/0272989X09336144. - DOI - PubMed
1. Lord SJ, Irwig L, Simes RJ. When is measuring sensitivity and specificity sufficient to evaluate a diagnostic test, and when do we need randomized trials? Ann Intern Med. 2006;144(11):850–5. doi: 10.7326/0003-4819-144-11-200606060-00011. - DOI - PubMed
1. Sackett DL, Haynes RB. Evidence base of clinical diagnosis: The architecture of diagnostic research. BMJ. 2002;324(2):539–41. doi: 10.1136/bmj.324.7336.539. - DOI - PMC - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

ZA 687/1-1/Deutsche Forschungsgemeinschaft

LinkOut - more resources

Full Text Sources

[1] Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, Williams JW, Kunz R, Craig J, Montori VM, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336(7653):1106–10. doi: 10.1136/bmj.39500.677199.AE. - DOI - PMC - PubMed

[2] Schünemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, Williams JW, Kunz R, Craig J, Montori VM, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336(7653):1106–10. doi: 10.1136/bmj.39500.677199.AE. - DOI - PMC - PubMed

[3] di Ruffano LF, Hyde CJ, McCaffery KJ, Bossuyt PMM, Deeks JJ. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ. 2012;344:686. doi: 10.1136/bmj.e686. - DOI - PubMed

[4] di Ruffano LF, Hyde CJ, McCaffery KJ, Bossuyt PMM, Deeks JJ. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ. 2012;344:686. doi: 10.1136/bmj.e686. - DOI - PubMed

[5] Lijmer JG, Leeflang M, Bossuyt PMM. Proposals for a phased evaluation of medical tests. Med Decis Making. 2009;29(5):13–21. doi: 10.1177/0272989X09336144. - DOI - PubMed

[6] Lijmer JG, Leeflang M, Bossuyt PMM. Proposals for a phased evaluation of medical tests. Med Decis Making. 2009;29(5):13–21. doi: 10.1177/0272989X09336144. - DOI - PubMed

[7] Lord SJ, Irwig L, Simes RJ. When is measuring sensitivity and specificity sufficient to evaluate a diagnostic test, and when do we need randomized trials? Ann Intern Med. 2006;144(11):850–5. doi: 10.7326/0003-4819-144-11-200606060-00011. - DOI - PubMed

[8] Lord SJ, Irwig L, Simes RJ. When is measuring sensitivity and specificity sufficient to evaluate a diagnostic test, and when do we need randomized trials? Ann Intern Med. 2006;144(11):850–5. doi: 10.7326/0003-4819-144-11-200606060-00011. - DOI - PubMed

[9] Sackett DL, Haynes RB. Evidence base of clinical diagnosis: The architecture of diagnostic research. BMJ. 2002;324(2):539–41. doi: 10.1136/bmj.324.7336.539. - DOI - PMC - PubMed

[10] Sackett DL, Haynes RB. Evidence base of clinical diagnosis: The architecture of diagnostic research. BMJ. 2002;324(2):539–41. doi: 10.1136/bmj.324.7336.539. - DOI - PMC - PubMed

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Randomized test-treatment studies with an outlook on adaptive designs

Affiliations

Randomized test-treatment studies with an outlook on adaptive designs

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources