Inconsistencies in standard approximations for selection coefficients at loci affecting a polygenic character

Abstract

Inconsistencies exist in the standard expansions used to approximate selection coefficients for alleles at a locus underlying a quantitative character. Allelic (marginal) fitnesses obtained from expansions based on average excesses differ from allelic fitnesses obtained from expansions based on genotypic values. Similarly, W, the mean population fitness based on summing over either allelic or genotypic fitnesses usually differs mean population fitness obtained by averaging over the unrestricted phenotypic distribution. A consistent value of W requires no variation in genotypic values. If, as suggested by Nagylaki (1984), expansions are corrected for the decrease in phenotypic variance resulting from conditioning on the presence of a particular allele or genotype, inconsistencies still exist. Unless integral of W(z)[Vzp"(z) + zp'(z) + p(z)] dz = 0, where p(z) is the phenotypic probability density function, Vz the phenotypic variance, W(z) the fitness of phenotypic value z, the primes denote differentiation with respect to z, allelic fitnesses based on average effects differ from allelic fitnesses based on genotypic values. This condition must also be satisfied in order for either expansion to give a consistent W, as first shown by Nagylaki. For arbitrary W(z), this is satisfied if and only if phenotypes are normally distributed.