Application of early warning signs to physiological contexts: a comparison of multivariate indices in patients on long-term hemodialysis

Abstract

Early warnings signs (EWSs) can anticipate abrupt changes in system state, known as "critical transitions," by detecting dynamic variations, including increases in variance, autocorrelation (AC), and cross-correlation. Numerous EWSs have been proposed; yet no consensus on which perform best exists. Here, we compared 15 multivariate EWSs in time series of 763 hemodialyzed patients, previously shown to present relevant critical transition dynamics. We calculated five EWSs based on AC, six on variance, one on cross-correlation, and three on AC and variance. We assessed their pairwise correlations, trends before death, and mortality predictive power, alone and in combination. Variance-based EWSs showed stronger correlations (r = 0.663 ± 0.222 vs. 0.170 ± 0.205 for AC-based indices) and a steeper increase before death. Two variance-based EWSs yielded HR95 > 9 (HR95 standing for a scale-invariant metric of hazard ratio), but combining them did not improve the area under the receiver-operating curve (AUC) much compared to using them alone (AUC = 0.798 vs. 0.796 and 0.791). Nevertheless, the AUC reached 0.825 when combining 13 indices. While some indicators did not perform overly well alone, their addition to the best performing EWSs increased the predictive power, suggesting that indices combination captures a broader range of dynamic changes occurring within the system. It is unclear whether this added benefit reflects measurement error of a unified phenomenon or heterogeneity in the nature of signals preceding critical transitions. Finally, the modest predictive performance and weak correlations among some indices call into question their validity, at least in this context.

Keywords: autocorrelation; biomarkers; critical transition; cross-correlation; mortality; network physiology; variance.

FIGURE 1

Histograms of index distributions organized by parameters on which they are based. Distributions are presented after indices were transformed and corrected for the number of observations included in the calculation. The number of calculated values was 4,550 for all indices except MAF_ac (n = 1,811), MAF_var (n = 1,811), and MMD (n = 3,756). As shown in Supplementary Figure S2, distributions further away from a normal one might impact the predictive power of the index.

FIGURE 2

Pairwise correlations among indices. Indices were categorized according to the parameter(s) they are based on. Xs represent correlations not significant at α = 0.05. Abbreviations: AC, autocorrelation; Var, variance. Variance-based indices are mostly highly correlated with one another, whereas AC-based indices show poor to moderate correlations. Correlations among different index categories are at best moderate, suggesting that they capture, at least in part, different biological signals.

FIGURE 3

Trend before death for each index. Indices were averaged by time window, and means were plotted along with the 95% confidence intervals. All indices were z-transformed and centered at 5 years before death for ease of comparison, and were split into two graphs for ease of visualization. Vertical dashed lines indicate the results from change point analyses, i.e., the break point in values trend. Note that change point analyses were performed using the indices calculated with a 3-month time window for a better time resolution, although the results are plotted here against the 6-month trends. The best performing indices are all variance-based, except MMD, which is also based on AC.

FIGURE 4

Mortality prediction for each index. A-C, HR95, i.e. the hazard ratio of being in the 97.5th percentile relative to the 2.5th percentile of the index, together with 95% confidence intervals are shown for each index in models including only this specific index ((A), n ranges from 2,306 to 7,931 for all indices except MMD_all, for which n = 46,276), models including all indices ((B), n = 2,270), and models including all indices except MAF_var and MAF_ac ((C), n = 7,499). All models control for age using a cubic spline (with 5 degrees of freedom), sex, diabetes diagnosis, and length of follow-up, clustering multiple observations per individual. Hues of blue represent variance-based indices, hues of purple represent indices based on variance and auto-correlation, hues of red represent indices based on auto-correlation, and green represents the index based on cross-correlation. (D) Receiver operating characteristic (ROC) curves for a basic model including only demographic and control variables (black; age, sex, diabetes diagnosis, and length of follow-up) and for models including selected predictive indices (i.e., CVPC1, Av_Var, and PC_var), alone or in combination, are shown. Values for the area under the ROC curve are indicated in parentheses for each model in the legend. These results show that, despite a very high correlation, CVPC1 and Av_Var each contribute to predict mortality when included in the same model (Panels (B,C)), indicating subtle nuances in the signal captured by these indices because of computation differences. Also, although not overtly powerful when taken alone, other indices such as PC_var, seem to improve mortality prediction when added to multivariate models (Panels (B–D)).