The opening 11 chapters of Sefer Breisheet can be read as explaining why there was a need for a Jewish people.
G-d created man in His image and tasked him with partnering with Him in conquering the world. G-d began the process of creation but instructed His partner to complete the job. Sadly, he – and even more so, his descendants – failed miserably. So much so that G-d was about the end the partnership, bringing the world back to tohu vebohu, emptiness and nothingness (Breisheet 6:7).
However, He soon found a potential replacement partner in Noach and tasked him once again with the command to “be fruitful and multiply and fill the earth” (Breisheet 9:1). This partnership faired a bit better but G-d saw it would be unable to excel. He thus chose a new partner, one with a more narrow mission, that of creating a (small) nation that would be a “kingdom of priests and a holy nation” (Shemot 19:6). Over time this nation would influence all other nations to recognize the One true G-d, making Avraham the eventual father of many nations – not just physically but eventually morally as well.
The remaining 39 chapters of Breisheet detail who will be part of this nation and who not. Lot, the nephew of the childless Avraham, appears to be the first candidate, but is ruled out due to his excessive focus on the material over the spiritual – hence his move to the lush area of Sedom despite its people being “bad and sinners to G-d very much” (Breisheet 13:14).
Yishmael too is rejected though it is less clear why. He does “make fun” (Breisheet 21:9) of his younger brother Yitzchak but other than that appears to be a most worthy candidate. As far as Avraham is concerned he is no less worthy than Yitzchak, and Avraham only “banished” him – both from his home and ultimately his nation – upon G-d’s directive.
In the next generation Eisav too is rejected, once again for reasons that are hard to deduce from the text. Here too, he was beloved by his father who wanted him to receive his special blessing. In the next generation it appears this pattern was to continue with Yaakov favouring Yosef and Yosef actively boasting about his special status. Yet just like his father and grandfather before him, Yaakov’s choice was incorrect. With no female influence – Rachel is dead and Leah is “missing” with no record of any relationship between the two – it will take G-d to correct Yaakov and begin turning a family into a nation. All of Yaakov’s children will be part of the covenant. The question to ask is why?
Clearly, the actions of the brothers seem far worse than those of Lot, Yishmael and Eisav. Throwing a brother into a pit, either leaving him to die or “just” selling him to slavery, is much worse than anything Eisav actually did. Eisav said he would kill his brother but he made no attempt to do so and hugged and kissed Yaakov upon seeing him. Even our Sages acknowledge the great honour Eisav bestowed upon his father – in contrast to the brothers who blatantly lied to theirs.
Add in sleeping with one’s father’s wife, wiping out a city, consorting with a prostitute and one wonders why all 12 brothers would be part of covenantal community. Would it not have made more sense to wait another generation and choose all the offspring of Menashe and Ephraim, the first brothers of whom there is no friction between them?
As important as this question is it seems that we rarely think about it, somehow taking it for granted this is how it was meant to be. And perhaps that is the best answer.
Objectively speaking the brothers – especially the four oldest – may not have been more worthy than Lot, Yishmael or Eisav. But choosing Yaakov’s children was G-d’s plan. Eventually, if there was to be a Jewish nation, more than one child would have to be included in the covenant – and who better to choose than one who has 12 sons?
Furthermore, one might argue that it was part of the divine plan to have our patriarchs father other nations.
Our patriarchs had a dual role. Not only were they to found our great nation, they were to be "a father for many nations". While G-d chose to make a special covenant with the Jewish people, the non-Jewish world has an important divine role. They too are created in G-d’s image and are to use their wisdom to better the world. They have much to teach the Jewish people.
Avraham thus fathers eight children, only one of whom would be part of the covenantal people. Yitzchak had to father only one additional son to complete the task. There was no “need” for Yaakov to be the father of any non-Jewish nation.
Additionally, the descendants of these nations have played pivotal roles – positive and negative – in the development of the Jewish people. Avraham, for example, fathered Midian, enabling Moshe to escape from Egypt, essentially being the incubator for Moshe Rabbeinu. Yitzchak fathered Eisav who in turn was the grandfather of Amalek, the eternal enemy of the Jewish people.
But it did not have to be this way. Yishmael could have, and likely would have, been part of the Jewish people, if not for how he was (mis)treated by Sarah and Avraham. The words of the Ramban are both prophetic and frightening. “Our mother [Sarah] did transgress by this affliction, and Avraham also by his permitting her to do so. And so, G-d heard her [Hagar’s] affliction and gave her a son who would be a wild-ass of a man, to afflict the seed of Avraham and Sarah with all kinds of affliction” (Ramban, Breisheet 16:6).
Eisav too could have been part of the Jewish people. Commenting on the verse “When the boys grew up”, (Breisheet 25:26) Rav Shimshon Raphael Hirsch notes that Yitzchak and Rivka made the fatal mistake of raising the twins in the same manner, oblivious the wisdom of Solomon that one must “educate a child according to their ways”. Eisav was a man of the field and needed outlets that were unnecessary for “a dweller in the tents”. If only they had realized so (and not played favourites) how different Jewish history might have been.
Out Sages teach that Yishmael did teshuva (Bava Batra 16b). "Ma'aseh avot siman lebanim, the events of the fathers indicate what will be with the children. Ultimately the children of Yitzchak and Yishmael will come together in peace – a prophecy that has begun to be fulfilled. May we soon witness with our own eyes the day that “nation shalt not lift sword against nation, neither shall they learn war anymore” (Yishayahu 2:4) and all the nations of the world will dwell in peace. May that day come soon.
 To be more precise it was the pre-covenantal Abram and Sarai who mistreated Hagar perhaps indicating they had a less developed character than the covenantal Avraham and Sarah.
 Ironically, the fact the Yitzchak loved Eisav and Rivka Yaakov, should have alerted them to the need to raise each according to their individual temperament.
 The Sages do not claim Eisav did teshuva, though there is such a view amongst our commentaries (see, for example, Chizkuni, Breisheet 25:8). Perhaps our Sages understood that as the ancestor of Amalek, the epitome of evil, he is not a candidate for teshuva but rather defeat and destruction. Yet here too our tradition includes the teaching that “the children of the children of Haman (a descendant of Amalek) taught Torah in Bnei Brak” (Gittin 57b).