Strong reproductive isolation between humans and Neanderthals inferred from observed patterns of introgression

Abstract

Recent studies have revealed that 2-3% of the genome of non-Africans might come from Neanderthals, suggesting a more complex scenario of modern human evolution than previously anticipated. In this paper, we use a model of admixture during a spatial expansion to study the hybridization of Neanderthals with modern humans during their spread out of Africa. We find that observed low levels of Neanderthal ancestry in Eurasians are compatible with a very low rate of interbreeding (<2%), potentially attributable to a very strong avoidance of interspecific matings, a low fitness of hybrids, or both. These results suggesting the presence of very effective barriers to gene flow between the two species are robust to uncertainties about the exact demography of the Paleolithic populations, and they are also found to be compatible with the observed lack of mtDNA introgression. Our model additionally suggests that similarly low levels of introgression in Europe and Asia may result from distinct admixture events having occurred beyond the Middle East, after the split of Europeans and Asians. This hypothesis could be tested because it predicts that different components of Neanderthal ancestry should be present in Europeans and in Asians.

Fig. 1.

Simulated landscape used in our simulations. The union of the dark green and brown zones represents the conventionally assumed Neanderthal range (22) (scenario A′ in Table 1), whereas the violet zone represents a larger range, including the Altai Mountains, where Neanderthals remains have been identified recently. The brown zone represents an even more restricted area of potential hybridization in the Middle East (scenario A′′ in Table 1). The gray zone is the Himalayan range, where migrations have been disallowed. The dark green dot is an arbitrary place of origin for the expansion out of Africa, and the two red dots are the locations of the two samples where introgression is measured (Paris, France and Beijing, China). In our simulations, the continental areas have been divided into square cells (cell size = deme area = 100 × 100 km²) where a human population and a Neanderthal local population could potentially coexist, compete, and exchange migrants.

Fig. 2.

Distribution of the proportion of simulations (among 10,000) resulting in Neanderthal introgression levels compatible with observations (1.9%–3.1%) (2) in both French and Chinese samples. Each likelihood curve corresponds to a different demographic scenario described in Table 1 and Fig. 1. Results were obtained by assuming a deme area of 100 × 100 km² (results obtained under a different map resolution are presented in Fig. S1). Solid lines correspond to scenarios that are equally likely (within 2 AIC units from the scenario with the highest likelihood), whereas scenarios shown with a dotted line have an associated AIC more than 2 units larger, and thus cannot be considered as equally well supported by the data. corr., correlated; lhood, likelihood; ME, Middle East; uncorr., uncorrelated.

Fig. 3.

Spatial distribution of introgression events having left traces in the French and the Chinese populations obtained from 10,000 simulations of scenario A and assuming an interbreeding success rate of 0.5%. The color scale represents the estimated density of the introgression events per deme.