Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2023 Jan 3;6(1):e2249321.
doi: 10.1001/jamanetworkopen.2022.49321.

Risk of Bias in Randomized Clinical Trials Comparing Transcatheter and Surgical Aortic Valve Replacement: A Systematic Review and Meta-analysis

Fabio Barili  1   2 , James M Brophy  3 , Daniele Ronco  4 , Patrick O Myers  5 , Miguel Sousa Uva  6   7 , Rui M S Almeida  8 , Mateo Marin-Cuartas  9 , Amedeo Anselmi  10 , Jacques Tomasi  10 , Jean-Philippe Verhoye  10 , Francesco Musumeci  11 , John Mandrola  12 , Sanjay Kaul  13 , Stefania Papatheodorou  1 , Alessandro Parolari  4 ; International Evidence Grading Research Initiative Targeting Transparency and Quality (INTEGRITTY)
Collaborators, Affiliations
Meta-Analysis

Risk of Bias in Randomized Clinical Trials Comparing Transcatheter and Surgical Aortic Valve Replacement: A Systematic Review and Meta-analysis

Fabio Barili et al. JAMA Netw Open. .

Abstract

Importance: Recent European Society of Cardiology/European Association for Cardio-Thoracic Surgery (ESC/EACTS) guidelines highlighted some concerns about the randomized clinical trials (RCTs) comparing transcatheter aortic valve implantation (TAVI) and surgical aortic valve replacement (SAVR) for aortic stenosis. Quantification of these biases has not been previously performed.

Objective: To assess whether randomization protects RCTs comparing TAVI and SAVR from biases other than nonrandom allocation.

Data sources: A systematic review of the literature between January 1, 2007, and June 6, 2022, on MEDLINE, Embase, and Cochrane Central Register of Controlled Trials was performed. Specialist websites were also checked for unpublished data.

Study selection: The study included RCTs with random allocation to TAVI or SAVR with a maximum 5-year follow-up.

Data extraction and synthesis: Data extraction was performed by 2 independent investigators following the PRISMA guidelines. A random-effects meta-analysis was used for quantifying pooled rates and differential rates between treatments of deviation from random assigned treatment (DAT), loss to follow-up, and receipt of additional treatments.

Main outcomes and measures: The primary outcomes were the proportion of DAT, loss to follow-up, and patients who were provided additional treatments and myocardial revascularization, together with their ratio between treatments. The measures were the pooled overall proportion of the primary outcomes and the risk ratio (RR) in the TAVI vs SAVR groups.

Results: The search identified 8 eligible trials including 8849 participants randomly assigned to undergo TAVI (n = 4458) or SAVR (n = 4391). The pooled proportion of DAT among the sample was 4.2% (95% CI, 3.0%-5.6%), favoring TAVI (pooled RR vs SAVR, 0.16; 95% CI, 0.08-0.36; P < .001). The pooled proportion of loss to follow-up was 4.8% (95% CI, 2.7%-7.3%). Meta-regression showed a significant association between the proportion of participants lost to follow-up and follow-up time (slope, 0.042; 95% CI, 0.017-0.066; P < .001). There was an imbalance of loss to follow-up favoring TAVI (RR, 0.39; 95% CI, 0.28-0.55; P < .001). The pooled proportion of patients who had additional procedures was 10.4% (95% CI, 4.4%-18.5%): 4.6% (95% CI, 1.5%-9.3%) in the TAVI group and 16.5% (95% CI, 7.5%-28.1%) in the SAVR group (RR, 0.27; 95% CI, 0.15-0.50; P < .001). The imbalance between groups also favored TAVI for additional myocardial revascularization (RR, 0.40; 95% CI, 0.24-0.68; P < .001).

Conclusions and relevance: This study suggests that, in RCTs comparing TAVI vs SAVR, there are substantial proportions of DAT, loss to follow-up, and additional procedures together with systematic selective imbalance in the same direction characterized by significantly lower proportions of patients undergoing TAVI that might affect internal validity.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Myers reported serving as secretary general elect for the European Association for Cardio-Thoracic Surgery and secretary for the Cardiothoracic Surgery Network. Dr Anselmi reported receiving personal fees from Abbott Cardiovascular outside the submitted work. Dr Kaul reported receiving personal fees from Boehringer-Ingelheim, NovoNordisk, Bayer, GSK, Abbott Consulting, and AstraZeneca outside the submitted work; and holding equity interest in Johnson & Johnson. No other disclosures were reported.

Figures

Figure 1.
Figure 1.. Forest Plot of Risk Ratio of Deviation From Assigned Treatment (DAT) in Transcatheter Aortic Valve Implantation (TAVI) vs Surgical Aortic Valve Replacement (SAVR) (Selective DAT) in Randomized Clinical Trials That Performed As-Treated or Modified Intention-to-Treat Analysis
There is a selective pattern characterized by a lower proportion of DAT in the TAVI group resulting in a 6.2-fold lower proportion of DAT in the TAVI group compared with the SAVR group. Only the NOTION (Nordic Aortic Valve Intervention) Trial does not show a selective DAT. The size of the solid squares is proportional to the weight of each study, the horizontal bars indicate the 95% CI for each study, and the diamond represents the pooled estimate with 95% CI. PARTNER 3 indicates Placement of Aortic Transcatheter Valve Trial 3; RE, random effect; and SURTAVI, Surgical Replacement and Transcatheter Aortic Valve Implantation.
Figure 2.
Figure 2.. Forest Plot Presenting the Proportion Meta-analysis of Patients Lost to Follow-up per Year of Follow-up
On average, 4.8% of patients at risk were missing at each follow-up time, with a progressive increase of proportion of patients lost to follow-up with ongoing follow-up, as shown by the increasing rates among subgroups of follow-up time. Diamonds indicate the pooled proportions with 95% CIs; the vertical dashed line represents the pooled overall proportion. NOTION indicates Nordic Aortic Valve Intervention Trial; PARTNER, Placement of Aortic Transcatheter Valve Trial; SURTAVI, Surgical Replacement and Transcatheter Aortic Valve Implantation; and TAVI, transcatheter aortic valve implantation.
Figure 3.
Figure 3.. Meta-regression of Association Between Loss to Follow-up and Follow-up Time
There is a significant association between proportion of patients lost to follow-up and follow-up time. Shaded area indicates 95% CI; diagonal line indicates the linear association between loss to follow-up and follow-up time; circles indicate the rates of loss to follow-up at follow-up time for each randomized clinical trial; and circle size indicates the weight of single data.
Figure 4.
Figure 4.. Forest Plot Presenting the Selective Risk of Loss to Follow-up
There is a selective pattern characterized by a significant 2.56-fold lower risk of dropouts at follow-up for transcatheter aortic valve implantation (TAVI). The selective loss to follow-up appears to decrease with increasing follow-up time. The size of the solid squares is proportional to the weight of each study, the horizontal bars indicate the 95% CI for each study, the diamonds represent the pooled estimates with 95% CIs, and the vertical dotted line indicates a risk ratio equal to 1. NOTION indicates Nordic Aortic Valve Intervention Trial; PARTNER, Placement of Aortic Transcatheter Valve Trial; RE, random effect; and SAVR, surgical aortic valve replacement.
Figure 5.
Figure 5.. Forest Plot Presenting the Risk Ratio of Patients Who Received Additional Treatments in Transcatheter Aortic Valve Implantation (TAVI) vs Surgical Aortic Valve Replacement (SAVR)
There is a selective pattern similar to deviation from assigned treatment and loss to follow-up, resulting in a 3.7-fold lower proportion of participants with additional procedures in the TAVI group. The size of the solid squares is proportional to the weight of each study, the horizontal bars indicate the 95% CI for each study, the diamond represents the pooled estimate with 95% CI, and the vertical dotted line indicates a risk ratio equal to 1. NOTION indicates Nordic Aortic Valve Intervention Trial; PARTNER, Placement of Aortic Transcatheter Valve Trial; RE, random effect; and SURTAVI, Surgical Replacement and Transcatheter Aortic Valve Implantation.
See this image and copyright information in PMC

References

    1. Mack MJ, Leon MB, Smith CR, et al. ; PARTNER 1 trial investigators . 5-Year outcomes of transcatheter aortic valve replacement or surgical aortic valve replacement for high surgical risk patients with aortic stenosis (PARTNER 1): a randomised controlled trial. Lancet. 2015;385(9986):2477-2484. doi:10.1016/S0140-6736(15)60308-7 - DOI - PubMed
    1. Gleason TG, Reardon MJ, Popma JJ, et al. ; CoreValve U.S. Pivotal High Risk Trial Clinical Investigators . 5-Year outcomes of self-expanding transcatheter versus surgical aortic valve replacement in high-risk patients. J Am Coll Cardiol. 2018;72(22):2687-2696. doi:10.1016/j.jacc.2018.08.2146 - DOI - PubMed
    1. Deeb GM, Reardon MJ, Chetcuti S, et al. ; CoreValve US Clinical Investigators . 3-Year outcomes in high-risk patients who underwent surgical or transcatheter aortic valve replacement. J Am Coll Cardiol. 2016;67(22):2565-2574. doi:10.1016/j.jacc.2016.03.506 - DOI - PubMed
    1. Makkar RR, Thourani VH, Mack MJ, et al. ; PARTNER 2 Investigators . Five-year outcomes of transcatheter or surgical aortic-valve replacement. N Engl J Med. 2020;382(9):799-809. doi:10.1056/NEJMoa1910555 - DOI - PubMed
    1. Leon MB, Smith CR, Mack MJ, et al. ; PARTNER 2 Investigators . Transcatheter or surgical aortic-valve replacement in intermediate-risk patients. N Engl J Med. 2016;374(17):1609-1620. doi:10.1056/NEJMoa1514616 - DOI - PubMed