It's largely because people are operating with a certain preconceived image in their head when they hear "transgender".
In reality, each and every one of us has been in a bathroom with someone who's transgender without even knowing it.
IE: Janet Mock (former editor of People magazine)
Janet has been likely using the women's restroom for quite some time with nobody even suspecting a thing.
Or, Shawn Stinson:
View attachment 175233
Think anyone has suspected anything when Shawn uses the men's room? doubtful
The reason for all the hysteria is because people have had this kind of image/notion planted in their head:
"Hi, my name is Georgina and I'm going to go watch your daughter change in the locker room"