I admit that it bothers me to an extent. Here's why:
1) It's always said before Christmas. I'm a bit hung up on accuracy. If it isn't Christmas or the twelve days of Christmas to Epiphany, then it ain't actually the Feast of the Nativity yet. Some of us actually fast before then and aren't supposed to make merry. I never say anything though, because it would just make me look petty.
2) It's always said in shops/restaurants by people wanting to make a point of some sort. I never get the sincere vibe anymore when I hear it. It's like it is a hostile challenge. I usually give a wan smile and respond, 'Yeah, you too.'
3) I wonder what Merry Christmas people would do when confronted with a proper liturgical answer/response scenario, as is found in an Orthodox parish? I much prefer this at church, where I know people are sincere (outside of family gatherings).
Person one: Christ is born!
Person two: Glorify Him!
ETA: It has occurred to me that I just have some lingering issues here. This is one of the things that bothered me when I was in my deist phase a few years back- and it still bothers me. I dislike forced cultural holidays and having to celebrate them in ways I don't want to celebrate just to placate my extended family, some of whom I'd rather not have to deal with at all, which would truly make my holidays happier. I don't like capitalist Christmas, Halloween, New Year's, Valentine's Day, Mother's Day, Father's Day, and Easter.
I don't decorate for Christmas either even though I don't have any problem with decorating. I just don't see the point. You just wind up having to take it all down again and waste money on it for no good reason. It doesn't stop my mother and my mother-in-law from giving me new Christmas decorations every year to add to a collection I don't use. I'm about ready to put the Twelve Days of Christmas china I never wanted, I've never used, and never intend to use on Craigslist. Heck- I'm at the point where Freecyle would do.
If only my parish celebrated Christmas on the Old Calendar like we do for Pascha! I'd rather just give money to charity, have a nice family dinner, and attend church- but every year I'm forced to exchange gift cards (which makes no logical sense as we all spend twenty-five dollars) with my cousins, and buy gifts for nieces and nephews- some of whom not only don't remember my name, but can't tell the difference between me and another aunt of theirs (and we look nothing alike- and they're too old to be ignorant). Don't get me started about having to deal with my co-workers and their love of forced participation at staff holiday parties. I'd get more out of giving to a charity in their name as my gift to them and would rather receive the same in return. I'm not rich, but I don't actually need anything and can't get worked up over pointless consumerism.