High reliability of morphokinetic annotations among embryologists

Abstract

Study question: Are morphokinetic measurements of time lapse-videos of human embryos comparable among operators?

Summary answer: There is little variation among morphokinetic measurements taken by different operators when analyzing the same time lapse-videos of human embryos.

What is known already: Morphokinetic analysis of preimplantation embryo development is a complementary method of embryo assessment increasingly used in IVF laboratories. Time-lapse videos of embryo development are normally viewed by trained embryologists and annotated with the times when specific developmental events occur. Such annotations form the basis of embryo selection algorithms, used to rank the embryos for transfer. It is unknown whether the reliability of morphokinetic annotations is related to the morphological characteristics of the analyzed embryo or to the ability of the embryologists performing the annotation. One study so far reported the reliability of morphokinetic annotations among three embryologists using the time-lapse system (TLS), but larger studies with different setups are needed to address this issue further.

Study design size duration: A prospective study was carried out between October 2015 and June 2016. Six embryologists with various degrees of experience in static, morphology-based evaluation, individually annotated the same 93 videos of preimplantation development, corresponding to 18 IVF/ICSI cycles, recorded with a TLS.

Participants/materials setting methods: Times of second polar body extrusion, appearance and disappearance of pronuclei, and embryo cleavages (times from 2-cell to 5-cell stage: t2, t3, t4, t5) were annotated. Each embryologist was blinded to the annotations of the others. Intra- and inter-observer agreement was evaluated by computing intra-class correlation coefficients (ICCs).

Main results and the role of chance: In the inter-observer analysis, most ICCs obtained were higher than 0.80, indicating a high level of agreement: t2: 0.93; t3: 0.80; t4: 0.89; t5: 0.89; disappearance of two pronuclei: 0.98. However, the ICCs obtained for second polar body extrusion and the appearance of two pronuclei annotations was lower: 0.51 and 0.63, respectively, indicating an average level of agreement. The ICCs obtained from the intra-observer analysis were also higher than 0.80 (t2: 0.96; t3: 0.89; t4: 0.88; t5: 0.86; disappearance of two pronuclei: 0.96). The ICCs obtained from second polar body extrusion and the appearance of two pronuclei annotations were 0.77 and 0.66, respectively. These results indicate that developmental timings, annotated in time-lapse videos, are highly reliable both within and among observers.

Limitations reasons for caution: The events at the developmental stages from 6-cells to blastocyst were not evaluated; since some morphokinetic algorithms use times past the 6-cell stage in their calculations, further studies should be carried out to understand the variations among observers in these cases.

Wider implications of the findings: Time-lapse measurement should be as objective as possible, especially for the first embryo cleavages, because they are often measured to define algorithms to assess the embryonic implantation potential. Our results show that measurements using this particular TLS are consistent and reliable both within and among operators.

Study funding/competing interests: None.

Trial registration number: Not applicable.

Keywords: embryo preimplantation development; embryo-selection algorithm; intra-class correlation coefficients; morphokinetic; reliability; time-lapse.