In any event, I think the potter metaphor itself really does put your position on shaky footing. Consider this passage from the Old Testament:
5Then the word of the Lord came to me saying, 6"Can I not, O house of Israel, deal with you as this potter does?" declares the Lord. "Behold, like the clay in the potter's hand, so are you in My hand, O house of Israel.
It really does stretch credulity to imagine that someone would use the very distinctive image of God, as a potter, fashioning a set of people, represented as pots, to denote a relationship in which God actually has no active role in shaping the people.
This is really asking too much - one cannot come with a metaphor that places more emphasis on the active role of God as the potter metaphor. Clay is lifeless and inert - the work of fashioning the pot is entirely that of the potter. So as much as I sympathize with the spirit of your position, I do not see how it can work Biblically.
5Then the word of the Lord came to me saying, 6 "Can I not, O house of Israel, deal with you as this potter does?" declares the Lord. "Behold, like the clay in the potter's hand, so are you in My hand, O house of Israel.
Yes, God can deal with Israel as the potter does with his clay. But, does God actually deal with Israel as the potter does? No, He does not.
Romans 9:20-24
But indeed, O man, who are you to reply against God? Will the thing formed say to him who formed it, “Why have you made me like this?” 21 Does not the potter have power over the clay, from the same lump to make one vessel for honor and another for dishonor? 22 What if God, wanting to show His wrath and to make His power known, endured with much longsuffering the vessels of wrath prepared for destruction, 23 and that He might make known the riches of His glory on the vessels of mercy, which He had prepared beforehand for glory, 24 even us whom He called, not of the Jews only, but also of the Gentiles? nkjv
God certainly did have the power over all the "clay" of mankind. And He can show His wrath against them at any time He desires.
But God didn't actually want to show His wrath toward any of mankind. Instead of wrath, He wanted to show His mercy to all - Jew and Gentile, so that each of these Jews and Gentiles could repent and be saved.
2 Peter 3:9
The Lord is not slack concerning
His promise, as some count slackness, but
is longsuffering toward us, not willing that any should perish but that all should come to repentance. nkjv
God is longsuffering. He is merciful. He desires that all people come to repentance and be saved.
God desired for all people to come to repentance and salvation and so He sent His own Son to die for them, to forgive their sins, to make it possible for them to spend eternity with Him.
1 Timothy 2:3-7
For this is good and acceptable in the sight of God our Savior, 4 who desires all men to be saved and to come to the knowledge of the truth. 5 For there is one God and one Mediator between God and men, the Man Christ Jesus, 6 who gave Himself a ransom for all, to be testified in due time, 7 for which I was appointed a preacher and an apostle—I am speaking the truth in Christ and not lying—a teacher of the Gentiles in faith and truth. nkjv
People fall away from the faith and forfeit eternal life because of their own sins.
2 Timothy 2:19-21
Be diligent to present yourself approved to God, a worker who does not need to be ashamed, rightly dividing the word of truth. 16 But shun profane and idle babblings, for they will increase to more ungodliness. 17 And their message will spread like cancer. Hymenaeus and Philetus are of this sort, 18 who have strayed concerning the truth, saying that the resurrection is already past; and they overthrow the faith of some. 19 Nevertheless the solid foundation of God stands, having this seal: “The Lord knows those who are His,” and, “Let everyone who names the name of Christ depart from iniquity.” 19 Nevertheless the solid foundation of God stands, having this seal: “The Lord knows those who are His,” and, “Let everyone who names the name of Christ depart from iniquity.”20 But in a great house there are not only vessels of gold and silver, but also of wood and clay, some for honor and some for dishonor. 21 Therefore if anyone cleanses himself from the latter, he will be a vessel for honor, sanctified and useful for the Master, prepared for every good work. nkjv
The sinner [vessel of dishonor] may become righteous [a vessel of honor] by repenting of his sins. When he repents of his sins, the blood of Jesus Christ then makes him clean - free of sin. When cleansed by God, he becomes sanctified and therefore useful for God because he is now prepared spiritually to do the good works that God had prepared for him to do.
Vessels of dishonor do not have to remain as vessels of dishonor.