Not necessarily - even though it´s always been the most convenient and attractive option, for me.
Look, this is not an unsolvable problem. I just felt, I´d share the sort of small issues that I am dealing with in real life (as opposed to those big moral questions that are discussed in this forum, as though they played a part in our real lives: Should I murder this person? Should I steal this car? Should I have sex with my neighbour´s dog? Should I commit genocide? Should I torture my child?

).
As I mentioned in the OP, even without Mr. X these kind of socializing events aren´t my kind of thing. And more often than not, I simply just don´t participate.
Then again, I love the idea that people try to create something beautiful (even though I personally may not see the beauty in it), and I don´t like the idea that such efforts are met with indifference or rejection. It´s somewhat hard to credibly communicate your appreciation of a plan, along with saying "No, sorry, that´s not for me."
As a matter of fact, when all these plans for common social events in this team came up, I was open about my condition. I told them: "I would like you to understand that I don´t enjoy myself on parties and such. I´m not only introverted - I just seem to feel the more lonely the more people are around, and the more they are determined to enjoy themselves. I´m weird that way." They were very accepting and asked a few questions about it. A lady smiled and said: "Well, you can learn that.", and I smiled back and said: "I´m afraid I don´t even want to learn that."
Long story short: I´m pretty sure nobody will even feel rejected if I don´t go. So, that´s a perfectly viable option without any real downsides.
But I was interested: Do other people occasionally encounter such a significant mis-match with another person - and what are your strategies of dealing with it?