A statistical methodology for analyzing co-occurrence data from a large sample

Abstract

Determining important associations among items in a large database is challenging due to multiple simultaneous hypotheses and the ability to select weak associations that are statistically but not clinically significant. The simple application of the chi2 test among all possible pairs of items results in mostly inappropriate associations surpassing the traditional (alpha=.05, chi2=3.94) threshold. One can choose a stricter threshold to find stronger associations, but the choice may be arbitrary. We combined the volume test of Diaconis and Efron with a p-value plot to select a more rigorous and less arbitrary threshold. The volume test adjusts the p-value of the chi2-statistic. A plot of adjusted p-values (1 - p versus N(p)), where N(p) is the number of test statistics with a p-value greater than p, should be linear if there are no true associations. The point where the plot deviates from a line can be used as a threshold. We used linear regression to select the threshold in a reproducible fashion. In one experiment, we found that the method selected a threshold similar to that previously obtained by manually reviewing associations.

Figure 1

The plot of adjusted p values in the overall co-occurrence data

Figure 2

Plot of p values adjusted by the general volume test for the hypertension data

Figure 3

Plot of p values adjusted by the general volume test for the bipolar disorder data

Figure 4

Plot of p values adjusted by the general volume test for the depressive disorder data

Figure 5

Plot of p values from conditional volume tests for the hypertension data

Figure 6

Plot of p values adjusted by conditional volume test for the bipolar disorder data

Figure 7

Plot of p values adjusted by conditional volume test for the bipolar disorder data