The latter, and I single out non-Orthodox, because I think the only Orthodox who've ever called me brother have been close personal friends, in the same minor order I am in, or been related by a Sacrament (my kums sometimes call me brother, and they would fall under close personal friend too). Now that you say it I will be honest and say I can see a situation where it could sound creepy when a fellow Orthodox calls me brother, but that'd be unusual in my experience, and the creepy factor would have to be larger than the reality that, by being in the same Church, the same Communion, we do have an existing relationship.
The difference though is that everytime I've been addressed as brother by a non-Orthodox, "Well see, brother..." "Oh c'mon now brother..." "Brother ______, are you saying...?" it's been by someone who is trying to convince me that Orthodox is wrong. If everytime you met someone who called you 'Slugger' outside your family they tried to convince you that you should buy something from them, you'd become wary of people who called you slugger.
But I think, as I write this, we're talking about two different situations. I mean it sounds creepy (which might not be the best word. I used it to mean 'it automatically makes me put my guard up') when someone verbally (or on a forum) addresses me as "brother." I think you might be more concerned with mentally believing there is a familial relationship there? That I don't find creepy, I think it's a response of your ecclessiology. It makes sense for you to believe that. I don't, personally, but that's more along the lines of my believing in...I don't know, any other doctrine that Orthodox believe and Anglicans don't. That doesn't sound insincere.
Am I making sense? It's more the verbal usage of the relationship-term as a title that I mistrust. The fact that others believe we are in that relationship doesn't bother me, it's just another difference in our traditions. Those people who actually regard all Christians as brother seem to not use the term (getting back to what I had mentioned before). It is, again in my experience, those who don't actually believe in the "family of all Christians" who are more apt to verbally call me "brother" in an attempt to bring me into their church. That's what I meant by 'insincere' too.
And I really do recognize that if I lived in the south I might have to alter my way of thinking. I understand they use the term more commonly down there, whereas where I live it is not common for Protestants to call one another "Brother" even within their own church.