No common denominator: a review of outcome measures in IVF RCTs

Abstract

Study question: Which outcome measures are reported in RCTs for IVF?

Summary answer: Many combinations of numerator and denominator are in use, and are often employed in a manner that compromises the validity of the study.

What is known already: The choice of numerator and denominator governs the meaning, relevance and statistical integrity of a study's results. RCTs only provide reliable evidence when outcomes are assessed in the cohort of randomised participants, rather than in the subgroup of patients who completed treatment.

Study design, size, duration: Review of outcome measures reported in 142 IVF RCTs published in 2013 or 2014.

Participants/materials, setting, methods: Trials were identified by searching the Cochrane Gynaecology and Fertility Specialised Register. English-language publications of RCTs reporting clinical or preclinical outcomes in peer-reviewed journals in the period 1 January 2013 to 31 December 2014 were eligible. Reported numerators and denominators were extracted. Where they were reported, we checked to see if live birth rates were calculated correctly using the entire randomised cohort or a later denominator.

Main results and the role of chance: Over 800 combinations of numerator and denominator were identified (613 in no more than one study). No single outcome measure appeared in the majority of trials. Only 22 (43%) studies reporting live birth presented a calculation including all randomised participants or only excluding protocol violators. A variety of definitions were used for key clinical numerators: for example, a consensus regarding what should constitute an ongoing pregnancy does not appear to exist at present.

Limitations, reasons for caution: Several of the included articles may have been secondary publications. Our categorisation scheme was essentially arbitrary, so the frequencies we present should be interpreted with this in mind. The analysis of live birth denominators was post hoc.

Wider implications of the findings: There is massive diversity in numerator and denominator selection in IVF trials due to its multistage nature, and this causes methodological frailty in the evidence base. The twin spectres of outcome reporting bias and analysis of non-randomised comparisons do not appear to be widely recognised. Initiatives to standardise outcome reporting, such as requiring all effectiveness studies to report live birth or cumulative live birth, are welcome. However, there is a need to recognise that early outcomes of treatment, such as stimulation response or embryo quality, may be appropriate choices of primary outcome for early phase studies.

Study funding/competing interests: J.W. is funded by a Doctoral Research Fellowship from the National Institute for Health Research. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health. J.W. also declares that publishing research is beneficial to his career. J.W. and A.V. are statistical editors, and M.S. is Information Specialist, for the Cochrane Gynaecology and Fertility Group, although the views expressed here are not necessarily those of the group. D.R.B. is funded by the NHS as Scientific Director of a clinical IVF service. The authors declare no other conflicts of interest.

Keywords: CROWN; IMPRINT; IVF; assisted reproduction; core outcomes; infertility trial; live birth; ongoing pregnancy; outcome measures; reporting guidelines.

Figure 1

PRISMA diagram showing flow of studies in the review.

Figure 2

Reported live birth outcomes in 47 IVF RCTs in 2013–2014 by stage of intervention. Each row corresponds to a single study. Only studies reporting live birth outcome measures appearing in more than one study are shown (as opposed to all studies reporting live birth). Blue triangles () indicate that the study authors used a denominator that coincided with the point of randomisation in the trial. Red circles () indicate that the study authors did not use the point of randomisation as the denominator, but instead included only patients who reached a certain stage of treatment when calculating live birth rates, potentially undermining the random allocation in the study. Black squircles () indicate that it is unclear whether or not the denominator coincided with the point of randomisation.