Maximum likelihood inference of reticulate evolutionary histories

Abstract

Hybridization plays an important role in the evolution of certain groups of organisms, adaptation to their environments, and diversification of their genomes. The evolutionary histories of such groups are reticulate, and methods for reconstructing them are still in their infancy and have limited applicability. We present a maximum likelihood method for inferring reticulate evolutionary histories while accounting simultaneously for incomplete lineage sorting. Additionally, we propose methods for assessing confidence in the amount of reticulation and the topology of the inferred evolutionary history. Our method obtains accurate estimates of reticulate evolutionary histories on simulated datasets. Furthermore, our method provides support for a hypothesis of a reticulate evolutionary history inferred from a set of house mouse (Mus musculus) genomes. As evidence of hybridization in eukaryotic groups accumulates, it is essential to have methods that infer reticulate evolutionary histories. The work we present here allows for such inference and provides a significant step toward putting phylogenetic networks on par with phylogenetic trees as a model of capturing evolutionary relationships.

Keywords: incomplete lineage sorting; maximum likelihood; phylogenetic networks; reticulate evolution.

Fig. 1.

Phylogenetic networks. Here, the MRCA of A and B split from its MRCA with C, and some time after A and B split, hybridization occurred between B and C. Four independent loci, ▲, ●, ■, and ♦, are illustrated, for which a single individual is sampled from each of A and C and six individuals are sampled from B. Two gene trees are depicted for the ▲ and ♦ loci, and both trees agree in terms of their shapes. However, the disagreement of the species splitting pattern with the gene tree in red is due to ILS, whereas the disagreement with the gene tree in blue is due to hybridization. Furthermore, the ▲ locus exhibits no evidence of hybridization in B, the ♦ locus has lost all signal of vertical inheritance from the MRCA of B with A, and the other two loci exhibit varying degrees of hybridization signal in the population. Locus-specific inheritance probabilities are needed to capture such scenarios.

Fig. 2.

Accuracy of the method on simulated data. (A) Data were generated down the phylogenetic network ${\mathrm{\Psi }}_1$ (all internal branches, except for the horizontal edge, have lengths of 1 coalescent unit, and the inheritance probability is 0.1 for all loci). Results based on gene tree topology estimates (B) and gene tree topology and branch length estimates (C) are shown. For every number of loci, the rightmost bar corresponds to inference from the true gene genealogies and the other three bars, from left to right, correspond to gene genealogies estimated (using 100 bootstrap replicates and Eq. 3) from sequences of lengths of 250, 500, and 1,000, respectively. The dark blue, cyan, and yellow regions correspond to the number of times each of the networks ${\mathrm{\Psi }}_1$, ${\mathrm{\Psi }}_2$, and ${\mathrm{\Psi }}_3$, respectively, in A was inferred. The maroon region corresponds to the number of times any other network with a single reticulation was inferred. Here, one individual was sampled per taxon for each of the loci.

Fig. 3.

Optimal phylogenetic network inferred on the house mouse (M. musculus) dataset. A single individual was sampled from each of five populations: M. m. domesticus from France (DF), M. m. domesticus from Germany (DG), M. m. musculus from the Czech Republic (MZ), M. m. musculus from Kazakhstan (MK), and M. m. musculus from China (MC). The analysis found multiple, almost equally optimal, phylogenetic networks with two reticulation events. These multiple networks all agreed on the recipient populations but disagreed on the donor populations. One hybridization (the top dashed horizontal arrow) involves the MRCA of DF and DG as a recipient population, yet seems to have involved MK, MC, or their MRCA as the donor population. The second hybridization (the bottom dashed horizontal arrow) involves MZ as a recipient population, yet seems to have involved DF, DG, or their MRCA as the donor population. Branch lengths in coalescent units (on the tree branches) and inheritance probabilities (on the horizontal edges) are shown (full details of the data and results are provided in SI Appendix).