In john 6, Jesus clearly tells us some truths. That we DO NOT come to Him unless The Father has granted that to us.
So does God love everybody, but God only gives some people to His Son so that they are saved?
Why would God wait patiently for those He knows will not be coming to Christ because He has not given them to Christ?
The idea is more along that of Christ's parable of the wheat and the tares. Christ says, leave them both to grow together in God's field, the world, and at the end separate out the wicked from the just, lest by God showing His wrath against the wicked, some of the just suffer.
There are the family of God planted by the Son of Man in God's field the world. And there are the sons of disobedience, the sons of the wicked one, the devil, planted by the devil. There is none else.
35 And Jesus said to them, “I am the bread of life. He who comes to Me shall never hunger, and he who believes in Me shall never thirst. 36 But I said to you that you have seen Me and yet do not believe. 37 All that the Father gives Me will come to Me, and the one who comes to Me I will by no means cast out. 38 For I have come down from heaven, not to do My own will, but the will of Him who sent Me. 39 This is the will of the Father who sent Me, that of all He has given Me I should lose nothing, but should raise it up at the last day