Between- and Within-Cluster Spearman Rank Correlations

Abstract

Clustered data are common in practice. Clustering arises when subjects are measured repeatedly, or subjects are nested in groups (e.g., households, schools). It is often of interest to evaluate the correlation between two variables with clustered data. There are three commonly used Pearson correlation coefficients (total, between-, and within-cluster), which together provide an enriched perspective of the correlation. However, these Pearson correlation coefficients are sensitive to extreme values and skewed distributions. They also vary with data transformation, which is arbitrary and often difficult to choose, and they are not applicable to ordered categorical data. Current nonparametric correlation measures for clustered data are only for the total correlation. Here we define population parameters for the between- and within-cluster Spearman rank correlations. The definitions are natural extensions of the Pearson between- and within-cluster correlations to the rank scale. We show that the total Spearman rank correlation approximates a linear combination of the between- and within-cluster Spearman rank correlations, where the weights are functions of rank intraclass correlations of the two random variables. We also discuss the equivalence between the within-cluster Spearman rank correlation and the covariate-adjusted partial Spearman rank correlation. Furthermore, we describe estimation and inference for the three Spearman rank correlations, conduct simulations to evaluate the performance of our estimators, and illustrate their use with data from a longitudinal biomarker study and a clustered randomized trial.

Keywords: clustered data; nonparametric correlation measures; rank association measures.

FIGURE 1

Toy examples for the relationship between total (${\gamma }_t$), between‐cluster (${\gamma }_b$), within‐cluster (${\gamma }_w$) Spearman rank correlations, and the rank intraclass correlations (${\gamma }_{I_X}$, ${\gamma }_{I_Y}$). Subfigures (a)‐(f) correspond to 6 different data generating scenarios. For illustration purposes, we show five clusters and five (two for [f]) observations to represent the distribution of each cluster. Black dots represent cluster medians, and the other symbols represent the five clusters. The solid lines show the direction of the within‐cluster correlation and the dashed lines show the direction of the between‐cluster correlation.

FIGURE 2

Bias and coverage of 95% confidence intervals for our estimators of ${\gamma }_b$, ${\gamma }_w$, and ${\gamma }_t$ at different true values and different cluster sizes under Scenarios I (normality) and II (exponentiated $Y$). The circle sign stands for the approximation‐based estimator (${\widehat{\gamma }}_{b_A}$) of ${\gamma }_b$, and the plus sign stands for the cluster‐median‐based estimator (${\widehat{\gamma }}_{b_M}$) of ${\gamma }_b$. The number of clusters was set at 100. “1–50” means the cluster size follows a uniform distribution from 1 to 50. When $\left({\gamma }_b,{\gamma }_w,{\gamma }_t\right)=(0.79,-\mathrm{0.68,0.05})$ and cluster sizes are 10 or 1–50, ${\widehat{\gamma }}_{b_M}$ had negative bias greater than 0.05 and poor coverage below 75% (not shown in the figure due to out of bounds). For specific bias and coverage values of ${\widehat{\gamma }}_{b_A}$ and ${\widehat{\gamma }}_{b_M}$, please refer to Web Table 1.

FIGURE 3

Scatter plot of CD4 and CD8 counts (cells/mm³) and estimates of within‐, between‐cluster, and total Pearson (${\rho }_w$, ${\rho }_b$, ${\rho }_t$) correlations. The red cross sign represents the sample cluster median and the circle sign represents the observation. The estimates (95% confidence intervals) of within‐, between‐cluster, and total Spearman rank correlations are invariant to data transformations, ${\widehat{\gamma }}_w=0.53[\mathrm{0.51,0.55}]$, ${\widehat{\gamma }}_{b_M}=0.24[\mathrm{0.20,0.29}]$, ${\widehat{\gamma }}_{b_A}=0.21[\mathrm{0.17,0.26}]$, and ${\widehat{\gamma }}_t=0.29[\mathrm{0.25,0.32}]$, where ${\widehat{\gamma }}_{b_M}$ is the estimator of ${\gamma }_b$ based on cluster medians obtained from CPM and ${\widehat{\gamma }}_{b_A}$ is the estimator of ${\gamma }_b$ based on the linear approximation.

FIGURE 4

Scatter plots of PHQ‐9 scores, age at enrollment (years), HIV knowledge, HIV stigma, and 12‐month adherence (%) of female and male partners enrolled in the clustered randomized clinical trial. The red cross sign represents the sample cluster median and the dot sign represents the observation. The right side of each subfigure shows the estimates (95% confidence intervals) of within‐cluster, between‐cluster, and total Spearman rank correlations (${\widehat{\gamma }}_w$, ${\widehat{\gamma }}_{b_M}$, ${\widehat{\gamma }}_{b_A}$, and ${\widehat{\gamma }}_t$), where ${\widehat{\gamma }}_{b_M}$ is the estimator of ${\gamma }_b$ based on cluster medians obtained from CPM and ${\widehat{\gamma }}_{b_A}$ is the estimator of ${\gamma }_b$ based on the linear approximation.