It's all down to confidence.
In the beginning you will wear tight fitting clothes because it gives you that edge; later on down the road, you won't bother because you won't feel as if you need to - you'll know the physique is there, so you'll see it as more of a secret weapon, rather than something you need to show off all the time.
I for one gave up caring, and went on to focus on strength.
My mentality was, what's the point in being muscular if you can't even squat 300lb? and not only that, but without an 'assist' I wasn't going to get much bigger anyway, so strength was the new mountain to climb.
I ended up being able to outlift people who are twice my size in the end. That humbled them a bit.
I remember some lad came in an maxed out at 315 on the deadlift. I went in after him and warmed up with it for 10 reps. Fun times.
(Not hard though, when compared to some of the guys I've trained with in the past, my goodness.)