Coalescents and genealogical structure under neutrality

Abstract

Genealogical or coalescent methods have proved very useful in interpreting and understanding a wide range of population genetic data. Our aim is to illustrate some of the central ideas behind this approach. The primary focus is genealogy in neutral genetic models, for which the effects of demography can be separated from those of mutation. We describe the coalescent for panmictic populations of fixed size, and its extensions to incorporate various assumptions about variation in population size and nonrandom mating caused by geographical population subdivision. The effects of such genealogical structure on patterns and correlations in genetic data are discussed. An urn model is useful for simulating samples at loci with complex mutation mechanisms. We give two applications of the genealogical approach. The first concerns methods for estimating the mutation rate from infinitely-many-sites data, and the second relates to inference about recent common ancestors and population history.