Inferences of species phylogeny in relation to segregation of ancient polymorphisms

Abstract

Standard formulas of gene frequency change under genetic drift are used to derive the probability of obtaining incorrect phylogenetic information for three species due to segregation of ancient polymorphisms. This probability depends upon the level of polymorphisms at the time of speciation and is generally quite high unless the two speciation events are far apart in time. If phylogenetic data from multiple loci are available, a likelihood ratio test can be used to reject the null hypothesis in favor of the best phylogeny. The appropriate null hypothesis is either a trichotomy or an alternative phylogeny, depending on the data set. The likelihood ratios required for accepting the best phylogeny are given. These ratios are obtained by exact enumeration when the number of loci is small (n less than 15) and by an asymptotic approach for larger n's. In general, more than five loci are needed to resolve the species phylogeny.