Mathematical Constraints on FST: Biallelic Markers in Arbitrarily Many Populations

Abstract

[Formula: see text] is one of the most widely used statistics in population genetics. Recent mathematical studies have identified constraints that challenge interpretations of [Formula: see text] as a measure with potential to range from 0 for genetically similar populations to 1 for divergent populations. We generalize results obtained for population pairs to arbitrarily many populations, characterizing the mathematical relationship between [Formula: see text] the frequency M of the more frequent allele at a polymorphic biallelic marker, and the number of subpopulations K We show that for fixed K, [Formula: see text] has a peculiar constraint as a function of M, with a maximum of 1 only if [Formula: see text] for integers i with [Formula: see text] For fixed M, as K grows large, the range of [Formula: see text] becomes the closed or half-open unit interval. For fixed K, however, some [Formula: see text] always exists at which the upper bound on [Formula: see text] lies below [Formula: see text] We use coalescent simulations to show that under weak migration, [Formula: see text] depends strongly on M when K is small, but not when K is large. Finally, examining data on human genetic variation, we use our results to explain the generally smaller [Formula: see text] values between pairs of continents relative to global [Formula: see text] values. We discuss implications for the interpretation and use of [Formula: see text].

Keywords: FST; allele frequency; genetic differentiation; migration; population structure.

Figure 1

Bounds on $F_{\mathrm{ST}}$ as a function of the frequency of the most frequent allele, M, for different numbers of subpopulations K: (A) $K=2,$ (B) $K=3,$ (C) $K=7,$ (D) $K=10,$ and (E) $K=40.$ The shaded region represents the space between the upper and lower bounds on $F_{\mathrm{ST}}.$ The upper bound is computed from Equation 5; for each K, the lower bound is 0 for all values of M.

Figure 2

The mean $A\left(K\right)$ of the upper bound on $F_{\mathrm{ST}}$ over the interval $M\in \left[1/2,1\right),$ as a function of the number of subpopulations K. $A\left(K\right)$ is computed from Equation 8 (black line). The approximation $\stackrel{\sim }{A}\left(K\right)$ is computed from Equation 9 (gray dashed line). A numerical computation of the relative error of the approximation as a function of K, $\left|A\left(K\right)-\stackrel{\sim }{A}\left(K\right)\right|/A\left(K\right),$ finds that the maximal error for $2\le K\le 1000$ is 0.00174, achieved when $K=2.$ The x-axis is plotted on a logarithmic scale.

Figure 3

Joint density of the frequency M of the most frequent allele and $F_{\mathrm{ST}}$ in the island migration model, for different numbers of subpopulations K and scaled migration rates $4Nm$ (where N is the subpopulation size and m the migration rate): (A) $K=2,$ $4Nm=0.1;$ (B) $K=7,$ $4Nm=0.1;$ (C) $K=40,$ $4Nm=0.1;$ (D) $K=2,$ $4Nm=1;$ (E) $K=7,$ $4Nm=1;$ (F) $K=40,$ $4Nm=1;$ (G) $K=2,$ $4Nm=10;$ (H) $K=7,$ $4Nm=10;$ and (I) $K=40,$ $4Nm=10.$ The black solid line represents the upper bound on $F_{\mathrm{ST}}$ in terms of M (Equation 5); the red dashed line represents the mean $F_{\mathrm{ST}}$ in sliding windows of M of size 0.02 (plotted from 0.51 to 0.99). Colors represent the density of SNPs, estimated using a Gaussian kernel density estimate with a bandwidth of 0.007, with density set to 0 outside of the bounds. SNPs are simulated using coalescent software MS, assuming an island model of migration and conditioning on one segregating site. See Figure S5 in File S1 for an alternative algorithm for simulating SNPs. Each panel considers 100,000 replicate simulations, with 100 lineages sampled per subpopulation. Figures S2 and S3 in File S1 present similar results under finite rectangular and linear stepping-stone migration models.

Figure 4

${\overline{F}}_{\mathrm{ST}}/{\overline{F}}_{\max },$ the ratio of the mean $F_{\mathrm{ST}}$ to the mean maximal $F_{\mathrm{ST}}$ given the observed frequency M of the most frequent allele, as a function of the number of subpopulations K and the scaled migration rate $4Nm$ for the island migration model. Colors represent values of K. $F_{\mathrm{ST}}$ values are computed from coalescent simulations using MS for 10,000 independent SNPs and 100 lineages sampled per subpopulation. ${\overline{F}}_{max}$ is computed from Equation 11. Figure S4 in File S1 presents similar results under rectangular and linear stepping-stone migration models.

Figure 5

Mean $F_{\mathrm{ST}}$ values across loci for sets of geographic regions. Each box represents a particular combination of two, three, four, five, six, or all seven geographic regions. Within a box, the numerical value shown is $F_{\mathrm{ST}}$ among the regions. The regions considered are indicated by the pattern of “.” and “X” symbols within the box, with X indicating inclusion and “.” indicating exclusion. From left to right, the regions are Africa, Middle East, Europe, Central/South Asia, East Asia, Oceania, and America. Thus, for example, X...X.. indicates the subset {Africa, East Asia}. Lines are drawn between boxes that represent nested subsets. A line is colored red if the larger subset has a higher $F_{\mathrm{ST}}$ value, and blue if it has a lower $F_{\mathrm{ST}}.$ Computations rely on 577,489 SNPs from the HGDP.

Figure 6

$F_{\mathrm{ST}}$ values for sets of geographic regions as a function of K, the number of regions considered. (A) ${\overline{F}}_{\mathrm{ST}}$ computed using Equation 10. (B) ${\overline{F}}_{\mathrm{ST}}/{\overline{F}}_{\max }$ computed using Equation 11. For each subset of populations, the value of $F_{\mathrm{ST}}$ is taken from Figure 5. The mean across subsets for a fixed K appears as a solid red line, and the median as a dashed red line.

Figure 7

Joint density of the frequency M of the most frequent allele and $F_{\mathrm{ST}}$ in human population-genetic data, considering 577,489 SNPs. (A) $F_{\mathrm{ST}}$ computed for pairs of geographic regions. The density is evaluated from the set of $F_{\mathrm{ST}}$ values for all 21 pairs of regions. (B) $F_{\mathrm{ST}}$ computed among $K=7$ geographic regions. The figure design follows Figure 3.

Figure B1

The first and last local minima of $F_{\mathrm{ST}}$ as functions of the frequency M of the most frequent allele, for $K\ge 3$ subpopulations. (A) Relative positions within the interval $\left[i/K,\left(i+1\right)/K\right)$ of the first and last local minima, as functions of K. The position $x_{min}\left(i\right)$ of the local minimum in interval $I_i$ is computed from Equation B1. If K is odd, then this position is $x_{min}\left[\left(K-1\right)/2\right];$ if K is even, then it is $x_{min}\left(K/2\right).$ The position of the last local minimum is $x_{min}\left(K-2\right).$ Dashed lines indicate the smallest value for $x_{min}\left(i\right)$ of $1/2,$ and the limiting largest value of $2-\sqrt{2}.$ (B) The value of the upper bound on $F_{\mathrm{ST}}$ at the first and last local minima, as functions of K. These values are computed from Equation 5, taking $\lfloor KM\rfloor =i$ and $\left\{KM\right\}=x_{min}\left(i\right),$ with $x_{min}\left(i\right)$ as in part (A). Dashed lines indicate the limiting values of 1 and $2\sqrt{2}-2$ for the first and last local minima, respectively.