I think the intent of the word is what is the most significant, especially since there are variations in the meaning and colloquial usage between different cultures and generations. If someone has deliberately said or written something to be offensive, then it's natural and appropriate to then take proportional offense. In many cases, though, a person has used a word or term lightly and innocently, and the accusations, outrage, and condemnation that are hurled back can be disproportionately hurtful and unkind. As an example, back in 2012 there was a popular "about me" tag on YouTube, and a 14-year-old Texan "beauty guru" friend of mine responding to it answered the question about her personality by describing herself as a spaz. Most American viewers understood that she meant she's very excitable, quirky, and hyper. She was promptly assailed with accusations, primarily from British subs, that she was mocking those with cerebral palsy and should be deeply ashamed of herself. It actually spawned these response videos that called her out and shamed her as being callous and insensitive, or an ignorant idiot. A simple comment or heads up via message explaining without any accusation of wrongdoing or judgement that the word was considered to be offensive would have been far more kind and teachable. "Mad" is also used a bit differently in Britspeak vs. Americanism. The "crazy" connotation is usually one that has more affection to it, such as being "mad" / "crazy" about a person (adoration for him or her) or creatively eccentric in the Mad Hatter way. Girls from my dance studio who go to a local high school did a Mad Hatter themed Homecoming pep rally dance and had shirts about the "madness." None of it was to demean or in any way offend someone with a mental illness. In the spring we have "March Madness" to celebrate basketball playoffs, and there's clearly no intent to disparage anyone with the use of the word. In youth culture there's also words that are used differently, such as using "sick" as an adjective that means awesome. So I think context is imperative.
I'm not stating that we shouldn't be attentive to our words and sensitive and considerate to others because we absolutely should, but I do think we need to be reasonable. When a correction truly is necessary, it needs to be done in an appropriate way and with consideration to the context and the intent. Haranguing someone for a benign-intentioned faux pas is just as unkind if not more so than making the faux pas.