Obviously, God punishes non-christians. He does this because he intends to do it. He does it because it produces an achievement. Although the Bible says that God takes no pleasure that the wicked should die (Ezekiel 18:23), the Bible also says that it will please God to destroy the wicked:
Deuteronomy 28:15, 20-24, 63
If you do not obey the LORD your God and do not carefully follow all his commands and decrees I am giving you today, all these curses will come upon you and overtake you: The LORD will send on you curses, confusion and rebuke in everything you put your hand to, until you are destroyed and come to sudden ruin because of the evil you have done in forsaking him. Just as it pleased the LORD to make you prosper and increase in number, so it will please him to ruin and destroy you.
The Bible also says that the righteous will rejoice at the observation of this cursing:
Psalm 58:10-11 (King James Version)
The righteous shall rejoice when he seeth the vengeance: he shall wash his feet in the blood of the wicked. So that a man shall say, Verily there is a reward for the righteous: verily he is a God that judgeth in the earth.
The righteous in heaven shall rejoice, and they shall also laugh and mock:
Proverbs 1:24-26 (New International Version)
But since you rejected me when I called and no one gave heed when I stretched out my hand, since you ignored all my advice and would not accept my rebuke, I in turn will laugh at your disaster; I will mock when calamity overtakes you.'
In other words, it is not a sin to experience joy from witnessing the punishments of the non-Christians. Rather the opposite is true -- God commands us to have this emotional reaction:
Revelation 18:8, 20
In one day [Babylon's] plagues will overtake her: death, mourning and famine. She will be consumed by fire, for mighty is the Lord God who judges her. Rejoice over her, O heaven! Rejoice, saints and apostles and prophets! God has judged her for the way she treated you.
This emotional reaction is a distinguishing mark of God's very own people, as Jonathan Edwards says in the sermon,
"The End of the Wicked Contemplated by the Righteous":
The sight of this strict and immutable justice of God will render him amiable and adorable in their eyes. They will rejoice when they see him who is their Father and eternal portion so glorious in his justice.
Divine justice in the destruction of the wicked will then appear as light without darkness, and will shine as the sun without clouds, and on this account will they sing joyful songs of praise to God, as we see the saints and angels do, when God pours the vials of his wrath upon antichrist; Rev. 16:5-7 They sing joyfully to God on this account, that true and righteous are his judgments, Rev. 19:16. They seeing God so strictly just will make them value his love the more. . . .
It will occasion rejoicing in them, as they will have the greater sense their own happiness, by seeing the contrary misery. It is the nature of pleasure and pain, of happiness and misery, greatly to heighten the sense of each other. Thus the seeing of the happiness of others tends to make men more sensible of their own calamities; and the seeing of the calamities of others tends to heighten the sense of our own enjoyments. . . .
When they shall see how miserable others of their fellowcreatures are, who were naturally in the same circumstances with themselves; when they shall see the smoke of their torment, and the raging of the flames of their burning, and hear their dolorous shrieks and cries, and consider that they in the mean time are in the most blissful state, and shall surely be in it to all eternity ; how will they rejoice!
I suppose that we may not have the Bible in heaven, but if we didn't have it there, we would know what it is that the Bible said about God's purposes in punishing non-Christians: We would remember on a continual basis that he is punishing them without the intent of correcting them thereby. Because on this basis we understand the nature of God's holiness.