The catastrophe to the world with the global flood was necessarily with terrible things that were going on in the world, there was great wickedness coming from humanity, with apostasy of any who had been godly, save Noah. Genetic mutation came from corrupting processes over more than a thousand years among the gene pool of humanity, giantism would be something that certainly can result from it. The godly people were not necessarily solely descendants of Seth, and the text about this in scriptures does not clearly say that giants resulted directly from sons of God taking daughters of man, which are any women, for themselves.
It might be considered that judgment was coming on humanity, with the world that was providing for humanity perishing too. Angelic beings were not going to perish from this, with them being the ones who should be under judgment with their responsibility, if they were committing the evil being there taking human women for themselves, and somehow, with explanation of it wanting, having viable offspring with them.
You can have that answer for yourself, sure, but the Bible doesn't provide something conclusive for that. Yahweh God said that only 120 years would remain for man, at the time before God told Noah to build the ark, with giving the detailed instruction for it. There would be time for testimony to the wicked of humanity, including the godly who had apostasied. Or do you think they were not given time for that? In view of the information that there were many yet living their lifespans far beyond 120 years, still for many many centuries after, regardless of not walking with God, and the lifespans did not just decrease until they reached a span of 120 years and stop there, understanding is logical of the 120 years still left to man as that time for that great ark to be built and the time left for any to possibly repent, with the testimony they had, and join Noah, or perish. As brutal as judgment was going to be for them, it isn't like God to not give plenty of time to turn from sin that will have them under judgment, with this occasion.
This is also showing your own interpretation. I know some claim finds of giant skeletons of such superhuman size, but this is not confirmed with recognition from secular media or news sources. If the Nephilim were so much larger than the range for humans, you then do need an explanation from what would be supernatural for that. But such a different size is not stated from the Bible. There were such people of giant size long after the global flood too, you can be aware of, and the only size stated ever for giants is still in the range of humanity, though certainly with there being genetic alteration as an explanation.