Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Mar 3;8(3):e252152.
doi: 10.1001/jamanetworkopen.2025.2152.

Quantitative Bias Analysis for Single-Arm Trials With External Control Arms

Alind Gupta  1   2   3 , Grace Hsu  3 , Seamus Kent  4 , Stephen J Duffield  5 , Evie Merinopoulou  3 , Alexandre Lockhart  3 , Paul Arora  6   7 , Joshua Ray  8 , Samantha Wilkinson  8   9 , Nicolas Scheuer  10 , Sreeram V Ramagopalan  8 , Rolf H H Groenwold  11 , Sanjay Popat  12 , Miguel A Hernán  13
Affiliations

Quantitative Bias Analysis for Single-Arm Trials With External Control Arms

Alind Gupta et al. JAMA Netw Open. .

Abstract

Importance: Unmeasured confounding is a key concern for decision-makers when observational datasets are used to assemble external control arms (ECAs) for single-arm trials.

Objective: To investigate the utility of quantitative bias analysis (QBA) for exploring the sensitivity to unmeasured confounding of nonrandomized analyses using ECAs.

Design, setting, and participants: This study emulated 15 treatment comparisons using experimental arms from existing randomized trials in advanced non-small cell lung cancer (aNSCLC) conducted after 2011 and ECAs derived from observational data. Participants were eligible individuals diagnosed with aNSCLC between January 1, 2011, and March 1, 2020. After adjustment for measured baseline confounders, a prespecified QBA was conducted to address potential bias by known unmeasured and mismeasured confounders. The QBA relied on a synthesis of external evidence from a targeted literature search, randomized trial data, and clinician input. Hazard ratios from the original randomized trials were compared with those from their emulation based on ECA analyses. Analyses were completed from February 2022 to October 2023.

Exposure: Initiation of systemic therapies for aNSCLC.

Main outcomes and measures: Hazard ratios for all-cause death.

Results: Sample sizes varied from 52 to 830 depending on the treatment group. The mean difference in the log hazard ratio estimates when using the original control arm vs the ECA for each trial was 0.247 in unadjusted analyses (ratio of hazard ratios, 1.36), 0.139 when adjusted for measured confounders (ratio of hazard ratios, 1.22), and 0.098 when adding external adjustment for unmeasured and mismeasured confounders (ratio of hazard ratios, 1.17).

Conclusions and relevance: QBA was feasible and informative in ECA analyses in which residual confounding was expected to be the most important source of bias. These findings encourage further exploration of how QBA can help quantify the impact of bias in other settings and when using other data sources.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Hsu reported receiving grants from Roche during the conduct of the study. Dr Merinopoulou reported being a consultant for Roche during the conduct of the study. Dr Arora reported receiving personal fees from Cytel during the conduct of the study. Dr Lockhart reported receiving funding from Roche during the conduct of the study. Dr Ramagopalan reported receiving personal fees from Roche during the conduct of the study. Dr Scheuer reported holding shares in Roche, Novartis, and Alcon outside the submitted work. Dr Popat reported receiving personal fees from AbbVie, AhHeart, Amgen, Arcus Biosciences, AstraZeneca, Bayer, Blueprint, Bristol-Myers Squibb, Boehringer Ingelheim, Chugai Pharma, Daiichi Sankyo, Eisai, Elevation Oncology, Ellipses, EMD Serono, EQRx, Gilead, GSK, Guardant Health, IO Biotech, Janssen, Lilly, Merck KGaA, Mirati, MSD, Novocure, Novartis, Pfizer, PHarmaMar, Pierre Fabre, Regeneron, Roche, Sanofi, Takeda, and Turning Point Therapeutics outside the submitted work. Dr Hernán reported receiving personal fees from ADIA Lab, Cytel, and ProPublica during the conduct of the study; and grants from the National Institutes of Health and Veterans Health Administration outside the submitted work. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Hazard Ratio (HR) Estimates and 95% CIs
Data from analyses using the original control arms (dark blue) of the randomized trials and the ECAs without confounding adjustment (orange), with adjustment for measured confounders only after multiple imputation (light blue), and with adjustment for measured and simulated confounders (gray). ECA indicates external control arm.
Figure 2.
Figure 2.. Sensitivity of Results to Deviations From Missing at Random (MAR) Assumption for Performance Status in Each External Control Arm (ECA) Analysis
Hazard ratios (HRs) adjusted for measured confounders for a range of deviations from MAR (δ ≠ 0) in the imputation model for performance status. Colors represent difference in log HR estimated from ECA analysis vs that from the corresponding randomized trial. For reference, a change of Eastern Cooperative Oncology Group performance status of 1.0 approximately corresponds to a δ equal to 3.
See this image and copyright information in PMC

References

    1. Anderson M, Naci H, Morrison D, Osipenko L, Mossialos E. A review of NICE appraisals of pharmaceuticals 2000-2016 found variation in establishing comparative clinical effectiveness. J Clin Epidemiol. 2019;105:50-59. doi:10.1016/j.jclinepi.2018.09.003 - DOI - PubMed
    1. Beaver JA, Howie LJ, Pelosof L, et al. . A 25-year experience of US Food and Drug Administration accelerated approval of malignant hematology and oncology drugs and biologics: a review. JAMA Oncol. 2018;4(6):849-856. doi:10.1001/jamaoncol.2017.5618 - DOI - PubMed
    1. Mishra-Kalyani PS, Amiri Kordestani L, Rivera DR, et al. . External control arms in oncology: current use and future directions. Ann Oncol. 2022;33(4):376-383. doi:10.1016/j.annonc.2021.12.015 - DOI - PubMed
    1. Thorlund K, Dron L, Park JJH, Mills EJ. Synthetic and external controls in clinical trials—a primer for researchers. Clin Epidemiol. 2020;(12):457-467. doi:10.2147/CLEP.S242097 - DOI - PMC - PubMed
    1. Lash TL, Fink AK. Applying Quantitative Bias Analysis to Epidemiologic Data. Springer; 2009. doi:10.1007/978-0-387-87959-8 - DOI

Publication types