Your logic is truly astounding.
You don't know Abraham had many other children other than with Sarah? (Genesis 25:1-4)
So just because Abraham can indeed be considered our father, your strange logic makes you conclude that Isaac is therefore also our ancestor.
I take that as at least implied by Paul, when he writes: "For they are not all Israel who are of Israel, nor are they all children because they are the seed of Abraham; but, "In Isaac your seed shall be called. That is, those who are the children of the flesh, these are not the children of God; but the children of the promise are counted as the seed." God had promised Abraham:
“"As for Me, behold, My covenant is with you, and you shall be a father of many nations. "No longer shall your name be called Abram, but your name shall be Abraham; for I have made you a father of many nations.” (Ge 17:4-5 NKJV)
Indeed, Paul quotes that in his letter to the Christians in Rome, in the context of talking of those who are "of faith":
“For the promise that he would be the heir of the world was not to Abraham or to his seed through the law, but through the righteousness of faith. For if those who are of the law are heirs, faith is made void and the promise made of no effect, because the law brings about wrath; for where there is no law there is no transgression. Therefore it is of faith that it might be according to grace, so that the promise might be sure to all the seed, not only to those who are of the law, but also to those who are of the faith of Abraham, who is the father of us all (as it is written, "I have made you a father of many nations") in the presence of Him whom he believed — God, who gives life to the dead and calls those things which do not exist as though they did; who, contrary to hope, in hope believed, so that he became the father of many nations, according to what was spoken, "So shall your descendants be."” (Ro 4:13-18 NKJV)