The new species has been called Australopithecus deyiremeda, which means "close relative" in the language spoken by the Afar people.
The ancient remains are thought to belong to four individuals, who would have had both ape and human-like features..
Living with Lucy
Lead researcher Dr Yohannes Haile-Selassie, curator of physical anthropology at the Cleveland Museum of Natural History in the US, told BBC News: "We had to look at the detailed anatomy and morphology of the teeth and the upper and lower jaws, and we found major differences.
"This new species has very robust jaws. In addition, we see this new species had smaller teeth. The canine is really small - smaller than all known hominins we have documented in the past."
The age of the remains means that this was potentially one of four different species of early humans that were all alive at the same time.
The most famous of these is Australopithecus afarensis - known as Lucy - who lived between 2.9-3.8m years ago, and was initially thought to be our direct ancestor.
However the discovery of another species called Kenyanthropus platyops in Kenya in 2001, and of Australopithecus bahrelghazali in Chad, and now Australopithecus deyiremedaI, suggests that there were several species co-existing.
Some researchers dispute whether the various partial remains really constitute different species, particularly for A. bahrelghazali. But Dr Haile-Selassie said the early stage of human evolution was probably surprisingly complex.
"Historically, because we didn't have the fossil evidence to show there was hominin diversity during the middle Pliocene, we thought there was only one lineage, one primitive ancestor - in this case Australopithecus afarensis, Lucy - giving rise to the next.
"That hypothesis of linear evolution has to be revisited. And now with the discovery of more species, like this new one... you have another species roaming around.
"What this means is we have many species that could give rise to later hominins, including our own genus Homo."
Dr Haile-Selassie said that even more fossils need to be unearthed, to better understand the path that human evolution took.
He added that finding additional ancient remains could also help researchers examine how the different species lived side-by-side - whether they mixed or avoided each other, and how they shared food and other resources in their landscape.