How can we account for the fact that Neanderthals had bigger cranial capacity than Homo Sapiens but nonetheless they were eventually eliminated? Bigger cranial capacity is correlated with higher mental abilities, thus we are led into believing the Neanderthals were, on the average, smarter than us. How it is than that our ancestors managed to gradually eliminate them?

Neanderthals and Homo Sapiens are two different species that both evolved from Homo Erectus. There was a long debate over the question whether Neanderthals were indeed a different species than Homo Sapiens. Some claimed that, in fact, Neanderthals could crossbreed with Homo Sapiens and that modern humans are the result of this interbreeding. However, tests comparing Neanderthal and modern human mitochondrial DNA show too great a dissimilarity for Neanderthals to have contributed to the human mitochondrial genome. Thus, it seems settled that Neanderthals were indeed wiped out by the arrival of Homo Sapiens in Europe.

But the story is more complicated. Homo Sapiens and Neanderthals appeared from different Homo Erectus populations and inherited different kinds of technological culture. As it happened, Neanderthals inherited the more advanced culture. By "technological culture" it is meant particular techniques of stone toolmaking. Neanderthals had better stone tools.
Neanderthals spread mostly to Europe and in some parts of Asia. When Homo Sapiens first spread from Africa, around 100 000 years ago, they did not manage to enter Europe, presumably because of Neanderthals, and spread to Asia. However, around 45 000 years ago, when Homo Sapiens had developed the Cro-Magnon type of stone tools, they returned to Europe. This time Neanderthals apparently didn't stand a chance. What happened during that period of around 55 000 years?

Neanderthals were the first humans who buried their dead and they developed better stone tools. It was recently discovered that they also created art figures. However, during the same period of time Homo Sapiens developed much further and faster. Relative to Homo Sapiens' advances in stone tool making Neanderthals look like stagnating. In other words, although Neanderthals were definitely a great advance compared to Homo Erectus, their inventiveness was no match for the Homo Sapiens. But how do we resolve this with the fact that the Neanderthals were probably more intelligent than us?

We can restate the question more precisely: "How do we resolve this with the fact that the Neanderthal individual was probably more intelligent than a modern human individual?" The cranial capacity tells us something about the individuals' mental abilities, but it tells us almost nothing about the social aspects of life.
Studying the inventiveness of Homo Sapiens scientists have found that literally all the major innovations that have changed the way we live, from the use of fire, to agriculture, to writing etc., have developed only in a few places. For example agriculture appeared independently only in around seven places on the entire planet. All the rest of human populations that engaged in farming did it because they had learned it from somebody. Therefore, the most important aspect of inventiveness is not the ability to invent, but the ability to transmit and to preserve innovations.

This gives us an important clue to why Neanderthals failed in the competition with Homo Sapiens. One of the most important means by which innovations are preserved and transmitted is language. Neanderthals had language themselves. This was proven in 1983 when a Neanderthal hyoid bone was found at the Kebara Cave in Israel. The hyoid is a small bone that holds the root of the tongue in place, a requirement to human speech and, therefore, its presence seems to imply some ability to speak. Recent studies found that due to the physical characteristics of Neanderthals' hyoid and the fact that their larynx was stouter than that of modern man, the average note emitted by Neanderthals were high pitched and sharper than that of modern man. This contradicts the stereotype of Neanderthals having ape-like grunts. However, the base of the Neanderthal tongue was positioned higher in the throat, crowding the mouth somewhat. As a result, Neanderthal speech would most likely have been slow-paced and nasalized.

The overall conclusion would be that, although Neanderthals did have the ability to speak, they were capable of articulating only a smaller number of phonemes. Jared Diamond described this limitation using the following example: imagine how many words you could say if the only sounds you were able to make were a, u, c, p. Imagine trying to say "Trinity College is a fine place to work." All you could say were something like "Capupa Cappap up a cap capupap."

This means that in order to transmit certain information much longer propositions are required.
(Apes like the chimpanzee also have the same type of problem. They are able to understand language and they can even be taught how to use the sign language, but they cannot actually speak due to their improper larynx. It is also interesting to remark that the shape of our larynx, a feature that enables us to speak, also has disadvantages: we can choke with food.)

Thus, it seems that the ultimate reason behind Neanderthals extinction was not due to their mental capacities but to the shape of their larynx.