Explainable artificial intelligence to identify follicles that optimize clinical outcomes during assisted conception

Abstract

Infertility affects one-in-six couples, often necessitating in vitro fertilization treatment (IVF). IVF generates complex data, which can challenge the utilization of the full richness of data during decision-making, leading to reliance on simple 'rules-of-thumb'. Machine learning techniques are well-suited to analyzing complex data to provide data-driven recommendations to improve decision-making. In this multi-center study (n = 19,082 treatment-naive female patients), including 11 European IVF centers, we harnessed explainable artificial intelligence to identify follicle sizes that contribute most to relevant downstream clinical outcomes. We found that intermediately-sized follicles were most important to the number of mature oocytes subsequently retrieved. Maximizing this proportion of follicles by the end of ovarian stimulation was associated with improved live birth rates. Our data suggests that larger mean follicle sizes, especially those >18 mm, were associated with premature progesterone elevation by the end of ovarian stimulation and a negative impact on live birth rates with fresh embryo transfer. These data highlight the potential of computer technologies to aid in the personalization of IVF to optimize clinical outcomes pending future prospective validation.

Fig. 1. Contributory follicle sizes for clinical outcomes.

Normalized permutation importance values (mean ± SD) of follicle sizes (in mm) in treatment cycles averaged across all eleven clinics in the cross-validation protocol. The outcome variables are all oocytes (a), metaphase-II (MII) mature oocytes (b), two-pronuclear (2PN) fertilized zygotes (c), and high-quality blastocysts (d), respectively. The shading highlights follicle sizes that have at least 50% normalized importance.

Fig. 2. Contributory follicle sizes for mature oocytes stratified by patient subgroups.

Normalized permutation importance values (mean ± SD) in treatment cycles of follicle sizes (in mm) in ICSI treatment cycles where oocyte maturity was graded (n = 14,140), averaged across all eleven clinics in the cross-validation protocol. The outcome variable is mature metaphase-II (MII) oocytes in all panels. The shading highlights follicle sizes that have at least 50% normalized importance. In a first stratification approach, (a, b) represent patients that were ≤35 years old at the time of treatment (n = 5707), and >35 years of age (n = 4717), respectively. The data in (c, d) are stratified by those with a GnRH agonist (“long”; n = 5420) or GnRH antagonist (“short”; n = 3981) IVF suppression protocol, respectively. In all cases, only cycles were considered where an hCG trigger was administered.

Fig. 3. Contributory follicles on preceding days to trigger and explainability on the day of trigger.

a Normalized permutation importance values (mean ± SD) in treatment cycles of follicle sizes (in mm) in ICSI treatment cycles where oocyte maturity has been graded averaged across all eleven clinics in the cross-validation protocol. Three separate models were trained on treatment cycles where an ultrasound scan was available on the day of trigger (DoT) administration (n = 14,140), a day prior to trigger (DoT-1; n = 10,457), or two days prior to trigger (DoT-2; n = 9533). The figure represents the expected growth trajectory of the most important follicle sizes during ovarian stimulation approaching the DoT. b A beeswarm plot of “SHAP” values for one clinic in the cross-validation protocol, indicating the contribution to the predicted value of metaphase-II (MII) oocytes. The discrete color map corresponds to the count of follicles on the DoT administration in each treatment cycle in the clinic; yellow: no (zero) follicles, blue: 1–2 follicles, green: 3 or more follicles, of that particular size (in mm).

Fig. 4. Mature oocyte yield with different triggering criteria.

Percentage improvement in mature oocyte yield number of mature oocytes divided by the total follicle count on the day of trigger whilst meeting the (a) current typical follicle size threshold-based criteria of either 2 or 3 lead follicles or (b) according to the proportion of follicles sized within the optimal ranges. The data points reflect the relative difference in the medians between the groups that fulfilled and did not fulfill the respective criteria. The first rule (≥2 follicles ≥17 mm) showed no significant improvement (ns; p = 0.229) in mature oocyte yield according to the two-sided Mann-Whitney (M-W) U-test; all other comparisons were significant (p < 0.0001). Aside from the first threshold-based rule, all other rules in (a) showed significant yield improvement compared to patients not meeting the respective criterion by the M-W U-test (p < 0.0001). In (b), all range cut-offs ≥5% showed statistically significant improvements in mature oocyte yield when using 15–18 mm as the criterion (p < 0.0001). All range cut-offs ≥10% showed significant improvements in mature oocyte yield when using 13–18 mm as the criterion (p < 0.0001); ≥5% of follicles in the 13–18 mm range presented no significant improvement in mature oocyte yield (ns; p = 0.125).

Fig. 5. Impact of follicle size profile and elevated progesterone on live birth rate.

Partial dependence plots under logistic regression modeling (n = 9843) to demonstrate the impact of (a) the percentage of follicles sized 13–18 mm on the day of trigger (DoT) (OR: 1.03 (1.00 − 1.06) per 10 percentage points change; p = 0.048), and (b) mean follicle size on the DoT (OR: 0.95 (0.93 − 0.98) per 1 mm change; p =  0.001), on predicted live birth rate (LBR). Both models were adjusted for: age, total follicle count on the DoT, and type of trigger administered (hCG or GnRH agonist). The mean LBR and its 95% confidence interval (CI) are plotted with 100 bootstrapped simulations. c The association (mean and 95% CI) between increasing progesterone elevation on the DoT versus mature oocyte yield (n = 646) and LBR (n = 427). d The association (mean and 95% CI) between an increasing number of follicles sized >18 mm on DoT versus serum progesterone levels on the DoT (n = 994). Having at least 7 follicles of >18 mm in size presented a significant elevation in progesterone with reference to patients with <2 follicles >18 mm on the DoT using the two-way Dunnett’s multiple comparisons test reported with adjusted p values.