By taking away someone I love wouldn't that be punishment to me?
Yeah, that happened to David. God punished David by killing the child he conceived with Bathsheba when he committed adultery with her. I'm not going to say that God isn't going to do that or will do that, because I'm not God and would never want to speak for Him. All we have to go by is the Bible, and there are instances when God does something like that.
But I can tell you this: God's way is perfect (Psalms 18:30), and whatever He decides is perfectly just and holy. God is also a God of mercy, and will forgive you when you are truly sorry with a contrite heart. When you make a mistake or commit a sin, God doesn't want you to condemn yourself like Adam and Eve did when they hid from Him in the garden, nor does He want you to offer Him a burnt offering or sacrifice to atone for you sin. Psalms 51:16-17 says this. The sacrifices of God are a broken spirit: a broken and a contrite heart will He not despise. So, it's not for you to worry about what God will or will not do, because His judgement and ways are far above our judgement and ways; it's simply your job to work at obeying Him, day in and day out, confessing your sins when you make them, and understanding that He's faithful and just to forgive you of your sins. Take up your cross daily and follow after Christ (Luke 9:23). If we sin, we have an advocate with the Father, the Son Christ Jesus (1 John 2:1).
So, my advice is not to sweat it. Work to kill the old man everyday, but don't condemn yourself or others as a punishment for yourself because you've sinned. You have a guilty conscious, and rightfully so, as we all do, because we've all sinned and fallen short of God's glory (Romans 3:23), but all who believe are also saved by grace through faith in Christ, and that not of yourself, it is the gift of God (Ephesians 2:8-9). So relax, brother and have faith that God will save you from yourself and will grow you into who He wants you to be.