1 John 1:5-10 seems pretty clear that we should confess our sins so they don't linger in darkness. Our instinct is to hide them, but that becomes more and more dangerous. 'you are only as sick as your secrets'.
Now as to who you confess them to, different faiths have different practices. Some say you confess only to Christ and it is personal, others to a brother, others to a Priest. Christ does the forgiveness, where there's a third party I believe it is there as part of accountability. It's letting someone else in and admitting to another human being where we are, it is helpful. I come from a 12 step recovery/addiction background where our 5th step is 'admitted to ourselves, God and another human being the exact nature of our wrongs', so confession is helpful, it keeps me reflective and corrected, like making sure the course of a ship is not varying off by degrees. It's easy to kid or justify ourselves when we are alone or just 'in our heads'. Stuff in the dark likes to stay in the dark, as John said. I can get on my knees and pray and repent, and that's between God and me. Letting someone else know involves trust and is another way of saying 'hey, I am really serious about this, I want it gone, help me if you can.' And God works through people too, expresses His forgiveness and acceptance through others while we are here on this earth.
By the same token, who you confess to is important. If I confess to a weak brother who does not hold me accountable but instead helps me justify my sin, or who condemns me or who gossips my sin to others what have I done, what healing is there in that? None. This is why some traditions formally use a priest. I do not believe that priests absolve or forgive sin, Christ does that. But by the same token, Christ lives in any believer. So if I say to a brother 'your sins are forgiven', not I but Christ says that. And not I but Christ has already done that for them. I am really just here to communicate Christ's message. As would be a priest. A priest can offer good counsel and be grounded in prayer and the word. In my tradition, a priest is the equivalent of a spiritual doctor.
Likewise there are some people to whom we must confess. If I have stolen $1000 from someone and I confess it to God and continue on without returning it, have I repented of the theft? What do you think a brother or a priest would counsel me to do? My own conscience would convict me to confess the sin to the person I stole from and to return the money.
So I feel like there is generally a need to confess hidden sins and make amends if we can to those we have harmed 'except when to do so would injure them or others'. In recovery we seek counsel from others and from God before making this step, especially if things like infidelity have been involved. I don't think a 'general confession' is necessary or helpful unless you have harmed or deceived a large body of people, though the early church did have general confessions before they moved towards priests as the exclusive facilitators for reasons I have stated above.
I do believe that repentance and confession is necessary throughout the life of any Christian and a natural consequence of the Holy Spirit indwelling a body of flesh. I don't do it out of fear of not being forgiven, I do it to keep myself accountable and my track in accordance with God's will, because I want to and because I believe it's his Will to help me and heal me.
It's all gonna come out into the Light anyway one day, I have no doubt about that.