QuCo: quartet-based co-estimation of species trees and gene trees

Abstract

Motivation: Phylogenomics faces a dilemma: on the one hand, most accurate species and gene tree estimation methods are those that co-estimate them; on the other hand, these co-estimation methods do not scale to moderately large numbers of species. The summary-based methods, which first infer gene trees independently and then combine them, are much more scalable but are prone to gene tree estimation error, which is inevitable when inferring trees from limited-length data. Gene tree estimation error is not just random noise and can create biases such as long-branch attraction.

Results: We introduce a scalable likelihood-based approach to co-estimation under the multi-species coalescent model. The method, called quartet co-estimation (QuCo), takes as input independently inferred distributions over gene trees and computes the most likely species tree topology and internal branch length for each quartet, marginalizing over gene tree topologies and ignoring branch lengths by making several simplifying assumptions. It then updates the gene tree posterior probabilities based on the species tree. The focus on gene tree topologies and the heuristic division to quartets enables fast likelihood calculations. We benchmark our method with extensive simulations for quartet trees in zones known to produce biased species trees and further with larger trees. We also run QuCo on a biological dataset of bees. Our results show better accuracy than the summary-based approach ASTRAL run on estimated gene trees.

Availability and implementation: QuCo is available on https://github.com/maryamrabiee/quco.

Supplementary information: Supplementary data are available at Bioinformatics online.

Fig. 1.

Felsenstein’s zone simulation. (a) Each gene tree branch length is scaled by μ_s and/or μ_l; for example, the length of the terminal branch of C becomes ${\mu }_l{\tau }_1+{\mu }_s{\tau }_2$. Rates μ_s and μ_l are selected such that terminal branches of A and C in the unrooted gene tree have expected length l, and other branches have expected length s, as shown. (b) MAP gene trees estimated using MrBayes with simulations in Felsenstein’s zone can have large estimation error, especially when $l/s$ is high and sequence lengths (boxes) are short

Fig. 2.

Felsenstein’s zone quartet simulations comparing QuCo to ASTRAL (a) or Bucky-Quartet (b). Each box shows a combination of long l (rows) and short s branch lengths (columns), and colors delineate ILS level controlled by d. Each ribbon shows the improvement of QuCo over ASTRAL or BUCKy, all run on MrBayes gene trees. When the ribbon is patterned, BUCKy is better than QuCo

Fig. 3.

Branch length accuracy on Felsenstein’s zone simulations, showing the distribution of estimated branch length divided by true branch length for correctly estimated species tree (the number of such cases shown in each case). Lines show the four quartiles and the dot shows the mean. Each box corresponds to a value of s, combining all l values. See Supplementary Figure S5 for better resolution

Fig. 4.

Gene tree estimation error on Felsenstein’s zone simulations. Each dot corresponds to one model condition, with the x-axis showing the improvement in species tree accuracy by QuCo compared to ASTRAL and the y-axis showing the improvement in the average gene tree accuracy for all genes. The size of dots corresponds to the accuracy of ASTRAL species trees

Fig. 5.

Species tree topological error (mean, standard error) under anomaly zone simulations versus the number of genes

Fig. 6.

30-taxon dataset. Left: Comparison of the error rate of the species tree generated by running ASTRAL on IQ-Tree ML gene trees and QuCo on IQ-Tree quartet likelihoods with 200 and 500 genes of 30-taxon dataset. The x-axis shows deviation from clock represented by parameter α (inverse of the variance of rate multipliers). Each box is over 50 replicates