About Ishmael being the heir of Abraham, Genesis says:
18 And Abraham said to God, "If only Ishmael might live under your blessing!"
19 Then God said, "Yes, but your wife Sarah will bear you a son, and you will call him Isaac. [4] I will establish my covenant with him as an everlasting...