Heterozygote excess in small populations and the heterozygote-excess effective population size

Abstract

It has been proposed that effective size could be estimated in small dioecious population by considering the heterozygote excess observed at neutral markers. When the number of breeders is small, allelic frequencies in males and females will slightly differ due to binomial sampling error. However, this excess of heterozygotes is not generated by dioecy but by the absence of individuals produced through selfing. Consequently, the approach can also be applied to self-incompatible monoecious species. Some inaccuracies in earlier equations expressing effective size as function of the heterozygote excess are also corrected in this paper. The approach is then extended to subdivided populations, where time of sampling becomes crucial. When adults are sampled, the effective size of the entire population can be estimated, whereas when juveniles are sampled, the average effective number of breeders per subpopulations can be estimated. The main limitation of the heterozygote excess method is that it will only perform satisfactorily for populations with a small number of reproducing individuals. While this situation is unlikely to happen frequently at the scale of the entire population, structured populations with small subpopulations are likely to be common. The estimation of the average number of breeders per subpopulations is thus expected to be applicable to many natural populations. The approach is straightforward to compute and independent of equilibrium assumptions. Applications to simulated data suggest the estimation of the number of breeders to be robust to mutation and migration rates, and to specificities of the mating system.