Men sometimes gain too much power and attack women with a sense of ownership. Men then get into fights over women with the best genetics, and the woman always dies in those fights.
That leads to a shortage of good genetics for viability of the species. When the "trafficking" becomes extremely efficient, the population becomes controlled by predator-genetics which literally kill the non-predators, creating their own non-viability: shortage of women, combined with a "cannibalistic" culture.
Intelligence becomes rare as the population fiends upon itself. Meanwhile, a few intelligent people see this outcome and organize themselves with intelligent breeding. This is the root of Royalty's attention to breeding. After the predators literally slaughter the population, the remaining are non-viable over several generations. Some Royalty become known as gods, such as Isis. Women become leaders, due to their shortage and are then allowed to select their own mates.
So, while the population still exists, its arrangement is precisely opposite for genetic viability, because women are not given a choice (or are given a false choice due to being surrounded by only predators). The "royals" organize themselves in class-hierarchy while the remaining population dies off over several generations, due to non-viability of their children. The "royals" survive through intelligent mating practices, but do not follow a Genghis Khan-type of propagation.