Neandertal-human admixture and early Homo

Last week’s pre-publication of a paper addressing the alternative hypotheses of Neandertal-human admixture vs. ancient African population structure, with respect explaining the signal of Neandertal DNA in living humans, has generated a lot of great posts.

First, John Hawks has a wonderful piece in which he not only puts the new publication in context, but also provides additional empirical evidence of Neandertal admixture using the genomic data from Ötzi, the Tyrolean “ice man.” The implication is pretty clear:

You can imagine, we have carried out just about every comparison we can think that could explain this result as anything other than greater Neandertal ancestry. Aaron and I will be putting our manuscript on the arXiv as soon as we’ve both signed off on all the text and figures, hopefully this week. This is simple stuff, and I see no reason not to be open about it — anybody with the Ötzi data can immediately do the same thing.

If Ötzi had elevated Neandertal ancestry, it is quite possible, even likely, that earlier Upper Paleolithic/Mesolithic Europeans had an even greater Neandertal contribution.

Dienekes has several posts on the topic (here, here, here), including one that considers the possibility that admixture and ancient African population structure are at play.

Approaching ancient population genetics papers like this one is no small task, as often the terminology and models employed can appear intimidating to the uninitiated. Here is a rule of thumb for what I look at when reviewing such papers, almost all of which involve delving into the supplemental information attached to this paper:

Does the paper consider multiple demographic scenarios?
Demography is a big confounding factor in the interpretation of genetic data. Different demographic scenarios can produce similar patterns of genetic variation. This is one of the areas where I hope we will see a lot of collaboration between geneticists, archaeologists and paleoanthropologists, as each of these disciplines can provide independent lines of evidence to address demographic hypotheses. In the case of the Sankararaman et al. paper, they are explicitly interested in testing different demographic scenarios, outlined in Table 2.

Does the paper consider the impact of natural selection?
Natural selection is arguably the hardest factor to incorporate into standard testable models because it can act in many different ways (e.g. positive selection, background selection) and can produce different results depending on its timing and intensity. The Sankararaman et al. paper does a partial check for selection, testing whether linkage disequilibrium (LD) brought on by positive natural selection might explain the pattern of SNP variation they observe, concluding it does not.

Does the paper make explicit their estimate of mutation rate?
Mutation rate has long been a problem as old methods for assessing mutation rate were based on indirect methods of looking at long-term genetic divergence from key fossil-calibrated evolutionary divisions. More recently, methods have been developed that attempt to directly assess mutation rate by looking at genomic data from parent-offspring relationships. Unfortunately, these two methods have produced mutation rate estimates not entirely in accord with another. Sankararaman et al. look directly at the impact differing mutation rates have on their model and assess the robusticity of their model across different rates.

Does the paper incorporate empirical data?
As Hawks points out in his post (linked above), real genetic data is easy to come by these days. A model that is not validated in some way through the use of such empirical data is not trying as hard as it could. Sankararaman et al. test their model against real data drawn from the 1000 Genomes Project.

The Sankararaman et al. paper provides pretty compelling support for the hypothesis that Neandertal admixture is a real part of the human pattern of genetic variation. Ancient population structure within Africa (or outside Africa) might still play a role as well, but is not likely sufficient to fully explain the variation we observe.

The issue of Neandertals relationship to living humans is fascinating in its own right. But it also has significance for broader questions of human evolution in the Pleistocene. First, the increased amount of data available to address this question allows us to get closer to coming up with an answer for the fuzzy question of whether Neandertals and living humans are different species. If, and the data John Hawks presents in the link about would support this view, early Upper Paleolithic Europeans have significant Neandertal ancestry, the idea that the current pattern of genetic variation is the result of a small number of introgressive events between two separate species becomes harder to support. Instead, the view that Neandertals were a divergent population, more divergent than any two living human populations, but not a different species in the classic Biological Species Concept sense, has to become the null hypothesis.

If Neandertals are not a different species, though, what does this mean for speciation earlier in the Pleistocene? My primary area of interest, for example, is in the origin and dispersal of Homo in the Lower Pleistocene. This is a time period, as evidenced by the recent publication of new fossils from Northern Kenya, with significant disagreements about the evolutionary pattern at play. Most paleoanthropologists appear to have settled on a consensus that multiple species of early Homo, the result of an early radiation of the lineage, are present early in the Pleistocene. At a minimum, Homo habilis and Homo erectus are concurrent (excluding other hominin taxa, such as Au. boisei, or the South African Australopiths and Sediba), in this consensus view.

But if this is the case, what is the evolutionary scenario and time frame under which they became sufficiently differentiated to be considered separate species? We have good reason to believe that Neandertals were considerably isolated, to the point of legitimately being considered an allopatric, or at least parapatric, group relative to early humans in Africa. This isolation developed over an interval of as much as several hundred thousand years, aided by considerable environmental heterogeneity brought on by glacial cycles. Can we identify a similar framework for evolutionary differentiation in the Lower Pleistocene of Africa?

Possibly. But keep in mind that the evolutionary origin of Homo is typically linked to several factors that, at least on the surface, might work against the development of strong reproductive isolation and differentiation. These include:

– the transition towards larger body size, increased energetic efficiency, and increased range size
– the transition towards larger brains and a technologically-mediated ecological adaptation
– the transition towards a broader diet

Of course, there are several big caveats that need to be mentioned. First, we do not know where/what Homo comes from, with competing claims for Australopithecus garhi, Australopithecus sediba, Australopithecus africanus or other fossils/taxa. The relative lack of knowledge from the 2-3 million year window makes this a difficult question to fully address. Second, we do not know when Homo emerged, so in looking at the case of concurrent species of Homo at 1.9 million years, we do not know if we are talking about 100,000 years of divergence or 500,000 (or more) years of divergence within Homo.

Regardless, the increasing knowledge we have about evolutionary processes at the end of the Pleistocene can only aid us in addressing questions aimed at the beginning of the Pleistocene. The best comparative model we have for the evolution of humans is…the evolution of humans. The better we understand one part of that timeline, the better equipped we are to understand other parts of our evolutionary history.