Well, for a start, there is no mention of purgatory or infant baptism in the Bible, so we can discount those right away. If people want to believe in those, then they are free to make that choice. Whether believing in them will do them good in the long run remains to be seen.
The other doctrines that you list are mentioned in the Bible and therefore are more likely to be believable. The interesting thing about the Bible is that it says what it says and does not go into lengthy justifying explanations of why it is true. It takes the attitude of "take it or leave it" concerning whether people decide to believe it or not.
God did not stop Israel and Judah worshiping idols. They made that choice, despite the many warnings giving by the prophets He sent them. But, as is shown in Isaiah and Jeremiah, there are consequences for the choices the people made. The choice to adopt idolatry resulted in utter ruin, disease, famine and death for those who participated. It is interesting in Ezekiel's vision, the angel with the pen and paper was told to go around the city and put a mark on the foreheads of all those who mourned and saddened by the idolatry. When the destroying angel went through to kill all the idolators, he was told not to harm those who had a mark on their foreheads.
Because God never punishes the innocent with the guilty, although there are severe penalties for those who reject God and run after idols, there are very positive consequences for those who remain true to God and walk in His ways.
So, we are quite free to choose what we want to believe, just as long as we are prepared to take the consquences, positive or negative.