Permutation tests to assess sex differences in omics data

Abstract

It is common to sex-stratify analyses of omics data and to report effects as 'sex-specific' when they are significant in only one sex. However, when analysing hundreds or thousands of molecules, this approach will yield many spurious 'sex-specific' effects if not supported by significant interactions. I illustrate this problem using an RNA sequencing dataset showing almost no significant sex by treatment interactions, but where sex-stratified analyses yield hundreds of 'sex-specific' effects of treatment. These 'sex-specific' effects could be spurious or could be real but not show interactions due to low statistical power. To distinguish these possibilities, I describe permutation tests, which provide an intuitive way to determine if a pattern of observations differs from what would be expected due to chance. For this dataset, assigning sex at random often generates more 'sex-specific' effects than the real data, demonstrating that there is little evidence of sex differences. Next, I simulate an RNA sequencing dataset that includes genes modelled to have sex-specific effects of a condition. As expected, analysis of this simulated dataset yields both significant interactions and sex-specific effects in sex-stratified analyses. While stratified analyses detect a higher number of sex-specific effects than the analysis of interactions, they erroneously identify genes not modelled to show sex-specific effects more often than interactions. A permutation test confirms that the number of sex-specific effects observed in the simulated dataset is greater than expected due to chance. Permutation tests can be applied to omics studies of sex differences, simultaneously providing (i) a clear and simple demonstration of the problems of sex-stratified analyses, and (ii) additional evidence of sex-specific effects where these are present. R code is provided for permutations, simulations, and plots to visualize potential sex-specific effects, which can be adapted to other types of data.