I think that our culture, so saturated with "image", can be extremely harsh in this regard. It is easy in our culture to feel unworthy when the reality is that most other people feel the same way, and so there is no need to feel that way. However, that still doesn't answer the question about what to do about it.
Personally, I have found the film The Last Samurai to be a wonderful idea about this process of recovery and renewal. (It is rated-R for violence, though only because it is about war and it is tastefully done I think) The story is about an American Cavalry captain who was ordered to slaughter innocent native american women and children. With that horrible act weighing on his conscience he fell to drinking. Long story short, he eventually meets the Lord of a clan of Samurai who rehabilitates him and slowly teaches him how to find discipline, strength, and ultimately joy in life again.
Now, the film is in an eastern context, but I think that it functions more as an allegory or message relevant to the American context today. The core of the movie is about virtue and discipline. The ultimate point we add is that God forgives, but often we still don't forgive ourselves. The battle to forgive ourselves is something that God cannot help with; He will not force our hand. But, he can guide us to gain the strength to do so. I'd suggest putting your energy into a discipline like an art, sport, martial art, or intense service position. If you can become good at it, you may be able to gain the inner-strength you need to overcome your self-doubt and unworthiness.
Through all this, it is good to remember that there is nothing we can do to make God love us any less. With people, I think that it is much more a function of present actions and worth than past troubles. Most people understand that people can change. So if you can find strength for yourself, people will see that. The beauty is, though, that for many new acquaintances that you will make they won't see your past mistakes until after they see your present worth. At least, that's the way that I look at other people.
I wish you the best. Godbless and Godspeed in tribulation.
-Apollonian
20/M INTJ