Gene genealogy in three related populations: consistency probability between gene and population trees

Abstract

A genealogical relationship among genes at a locus (gene tree) sampled from three related populations was examined with special reference to population relatedness (population tree). A phylogenetically informative event in a gene tree constructed from nucleotide differences consists of interspecific coalescences of genes in each of which two genes sampled from different populations are descended from a common ancestor. The consistency probability between gene and population trees in which they are topologically identical was formulated in terms of interspecific coalescences. It was found that the consistency probability thus derived substantially increases as the sample size of genes increases, unless the divergence time of populations is very long compared to population sizes. Hence, there are cases where large samples at a locus are very useful in inferring a population tree.