Detecting recombination from gene trees

Abstract

In this article, a method is proposed for detecting recombination in the sequences of a gene from a set of closely related organisms. The method, the Homoplasy Test, is appropriate when the sequences are rather similar, differing by 1%-5% of nucleotides. It is effective in detecting relatively frequent recombination between a set of rather similar strains, in contrast to previous methods which detect rare or unique transfers between more distant strains. It is based on the fact that, if there is no recombination and if no repeated mutations have occurred (homoplasy), then the number of polymorphic sites, v, is equal to the number of steps, t, in a most-parsimonious tree. If the number of "apparent homoplasies" in the most-parsimonious tree, h = t-v, is greater than zero, then either homoplasies have occurred by mutation or there has been recombination. An estimate of the distribution of h expected on the null hypothesis of no recombination depends on Se, the "effective site number," defined as follows: if ps is the probability that two independent substitutions in the gene occur at the same site, then Se = 1/ps. Se can be estimated if a suitable outgroup is available. The Homoplasy Test is applied to three bacterial genes and to simulated gene trees with varying amounts of recombination. Methods of estimating the rate, as opposed to the occurrence, of recombination are discussed.