That's one probable answer. My dance teacher is a former world champion in swing, but he says that waltz is a struggle for him. Of course, he looks fantastic when he waltzes, so the word "struggle" is relative.
I am much better at swings and latins than smooth dances. I think part of it is the speed, but the bulk of the difference is the rhythm. A swing dance has clearly defined points in the move that let you "recover." If you get off balance in a West Coast turn, you know that there's an anchor step no more than 6 beats away. An East coast move has a rock step every 8 beats. Those places let you recover your bearings. Smooth dances like Waltz and FT (lesser extent) just flow from point to point w/o recovery points.
It depends on the step, too. I can do a Waltz twinkle all day long - it has places where you stop and turn - recovery points. A silver waltz left turning box will wipe me out in three repeats because the mistakes pile up.
Ok, my partner's right - I've become a dance snob!