Receiver operating characteristic (ROC) analysis of day 5 morphology grading and metabolomic Viability Score on predicting implantation outcome

Abstract

Purpose: Assessment of embryo viability is a key component of in vitro fertilization (IVF) and currently relies largely on embryo morphology and cleavage rate. In this study, we used receiver operating characteristic (ROC) analysis to compare the Viability Score (generated by metabolomic profiling of spent embryo culture media using near infrared (NIR) spectroscopy) to morphologic grading for predicting pregnancy in women undergoing single embryo transfer (SET) on day 5.

Methods: A total of 198 spent embryo culture media samples were collected in four IVF centers located in the USA, Europe and Australia. First, 137 samples (training set) were analyzed by NIR to develop an algorithm that generates a Viability Score predictive of pregnancy for each sample. Next, 61 samples (validation set) were analyzed by observers blinded to embryo morphology and IVF outcome, using the Day 5 algorithm generated with the training set. Pregnancy was defined as fetal cardiac activity (FCA) at 12 weeks of gestation.

Results: The Area Under the Curve (AUC) was greater for the metabolomic Viability Score compared to Morphology [Training set: 0.75 versus 0.55, p = 0.0011; Validation set: 0.68 versus 0.50, P = 0.021], and for a Composite score (obtained using a model combining Viability Score with morphologic grading), compared to morphology alone [0.74 versus 0.50, p = 0.004].

Conclusions: Our findings suggest that Viability Score alone or in combination with morphologic grading has the potential to be a better classifier for pregnancy outcome than morphology alone in women undergoing SET on day 5.

Fig. 1

ROC curves for the Viability Score (green) and morphologic grade (blue), for day 5 SET embryo culture media samples in the Training Set. The cutoff values corresponding to the indicated true positive and false negative rates are shown on the curves. AUC_ROC for Viability Score = 0.75; AUC_ROC for morphologic grade = 0.55; P = 0.0011

Fig. 2

ROC curves for the Viability Score (green) and morphologic grade (blue), for day 5 SET embryo culture media samples in the Validation Set. The cutoff values corresponding to the indicated true positive and false negative rates are shown on the curves. AUC_ROC for Viability Score = 0.65; AUC_ROC for morphologic grade = 0.50; P = 0.021

Fig. 3

A. ROC curves for the composite score (red), Viability Score (green) and morphologic grade (blue), for day 5 SET embryo culture media samples in the Validation Set. AUC_ROC for composite score = 0.74, AUC_ROC for Viability Score = 0.68; AUC_ROC for morphologic grade = 0.50. Composite score vs. morphologic grade, P = 0.004; Composite score vs. Viability Score, P = nonsignificant

Fig. 4

ROC curves for the Viability Score of embryos of different morphologic subsets for day 5 SET embryo culture media samples in the Validation Set. ROC curves for Viability Score of embryos of morphologic grade A (green line), grade A and B (blue line), grades A, B, and C (red line), and grades A,B,C, and D (black line) are shown. AUC_ROC for each group is also shown. There was no significant difference in Viability Score from between any of the subsets of morphologic grades