New statistical tests of neutrality for DNA samples from a population

Abstract

The purpose of this paper is to develop statistical tests of the neutral model of evolution against a class of alternative models with the common characteristic of having an excess of mutations that occurred a long time ago or a reduction of recent mutations compared to the neutral model. This class of population genetics models include models for structured populations, models with decreasing effective population size and models of selection and mutation balance. Four statistical tests were proposed in this paper for DNA samples from a population. Two of these tests, one new and another a modification of an existing test, are based on EWENS' sampling formula, and the other two new tests make use of the frequencies of mutations of various classes. Using simulated samples and regression analyses, the critical values of these tests can be computed from regression equations. This approach for computing the critical values of a test was found to be appropriate and quite effective. We examined the powers of these four tests using simulated samples from structured populations, populations with linearly decreasing sizes and models of selection and mutation balance and found that they are more powerful than existing statistical tests of the neutral model of evolution.