I think it should be pointed out here, from a straight man (me) if that makes any difference, that a lot of straight men can very well recognize that other men are good looking (I don't know that 'physically attractive' really makes sense in this context, since a straight man wouldn't be physically attracted to them to begin with, but could still recognize that they're good looking), but probably don't like to do so because it puts their own physical shortcomings front and center in their own minds. So it's not so much homophobia in those cases as a certain kind of self-loathing that expresses itself as hate and unacknowledged jealousy. Kind of like a weird version of the entitlement of the 'incel' community, but with the disgust and anger turned against fellow men rather than women. "That guy gets all the women" is Not-Gay™ for "That man is very attractive, and I am not, and that makes me feel bad, so ewww, get out of my brain, 'gay' thought. I hate that guy, and the stupid women who choose him instead of me because he's clearly better looking than I am."
We'll never tell you that, though. We can't even admit it to ourselves, but it's not because we're secretly gay (though it certainly could be in some cases; that does still happen); it's because so many are inadequate when measured against whoever they are measuring themselves against, and it's easier to deny that there's anything good about the guy than it would be to put down the fork and go to the gym, or turn off the video game and go get a job, etc.