In responding to this question, I am giving my understanding on this difficult question. My viewpoint doesn’t represent other Christians, church or denomination whether Ishmael is the father of the Israelites.
From a cultural viewpoint, Ishmael is the firstborn of Abraham. Hagar was elevated from concubine to one of the wives of Abraham. If Hagar and Ishmael were respectful to Abraham and Hagar, it is likely that Ishmael would have been the heir of Abraham. The son born to Sarah wouldn’t be the firstborn of Abraham, but one of the sons of Abraham. That would be the cultural or social norm.
Sadly because of Hagar response to Hagar which affected Ishmael, Sarah asked/demanded of Abraham to cast out/disinherited them. He consented to her demand but was grieved that he had to do it. He sent them away. That action led that Sarah’s son would be the heir of Abraham. There would be only one child in the family of Abraham and Sarah. Issac would be the heir of Abraham. It must be remembered that God promised to bless Ishmael because he is a son of Abraham. Ishmael isn’t cursed at all. He was to become a great nation. He is known as the father of the Ishmaelites.
From a cultural and spiritual viewpoint, the son born to Abraham and Sarah is a gift from God. Isaac was a promised child while Hagar was a natural child of a fertile woman. As stated early, since Hagar and Ishmael were cast out of Abraham’s household, leaving Isaac as the heir of Abraham. The lineage of Abraham passed through Isaac, even though he was the “second” of Abraham but the first son of Sarah.
God chose Sarah to be the wife whom the promises given to Abraham would be passed on through Isaac. God could have left Sarah barren, but he did not. God’s ways are mysterious.
For more perspectives: