What would they complain about? That the woman has too many muscles, and too much facial hair? That the guy has breast implants? Even if they look like a person of the opposite sex, if they have the same sex organs as everybody else, I don't think a reasonable person would complain.
For starters, I imagine the first and most obvious complaint would be that their sons would see breasts... there seems to be a really big hang-up about that.
We're talking about the same segment of the country that thought that we were on our way to hell in a handbasket because some people saw Janet Jackson's nipple during the halftime show...
A trans-woman in mens/boys shower room at the rec center would likely make their heads explode.