If sex and gender are different things, then I agree that whether you play in a men's or women's league should be determined by your chromosomes. The sex that was listed on your birth certificate when you were born. Although there still might be a grey area with intersex individuals, how to sort trans athletes would be relatively straightforward. Until medical science figures out how to change sex on the chromosomal and natural hormonal levels, when a man becomes a trans woman, she should still compete with men. Thus, eliminating the problem of biological men clobbering biological women on the playing field, taking their gold medals, stealing their scholarships, etc. I doubt as many people would mind trans women competing on men's teams as on women's teams.
With that being said, trans women might be concerned about being bullied or otherwise mistreated by their non-trans teammates. That's a separate problem that needs to be addressed and resolved. Nobody should be bullied for how they look, or feel on the inside, but I don't think the answer is for them to compete against female athletes.
However, there is an aspect of this that I haven't quite figured out yet. In what category should trans men compete? Biological women who transition are given testosterone. Testosterone, when increased in an individual through artificial means, is a performance enhancing drug. As such, giving them an unfair advantage over women who do not take testosterone. Perhaps not to the extent of being born male, but the effect isn't negligible. Those who are against doping might still find this to be problematic.
Another possibility might be to let trans athletes participate in whatever category they prefer where body contact is unlikely (swimming, for instance, rather than wrestling), but have a separate category of awards. A category that would let them establish trans records over time to compare and compete with other trans athletes. This way, even if a trans woman wins the race, for instance, the first biological female to cross the finish line would still win the gold. The trans woman's record would stand to be beaten (or not) in the future by other trans women, but not pose a threat to non-trans women's records or rankings.