How can this be if we have a February 24 every year?
Similarly, a long time ago Scandavian calendars had 31 days in February.
The Romans didn't number days, but dated by reference to specific points in the month: the Kalends, the Nones and the Ides. So February 23rd was 'six days before the Kalends of March' to the Romans. Prior to Julius Caesar's calender reform, this is when they added their intercalary month to right their year with the solar year, as February 23rd was an important festival called the Terminalia.
Subsequently in the Julian calender, this is where the leap day was placed - just after the 23rd, so was called the Bisextile day (meaning the second sixth day before the Kalends of March, or twice sixth). The Julian calender was out of synch with the solar year by about a day a century, roughly.
This way of dating the day remained in use into the mediaeval period, and in formal Latin long afterwards. So when the Papacy established the Gregorian calender to correct the Julian in the 16th century, they still wrote it in formal Latin style, and the subsequent adoption thereof by the countries of Europe either did so as well, or merely referenced the Gregorian Calender. This means that legally and formally, the leap day is the 24th of February in a Leap year.
The transition from Julian to Gregorian was uneven though, with Catholic countries quickly adopting it and Protestant ones doing so gradually over the next centuries. Britain did it in one go in 1752, suddenly dropping 11 days. Sweden did it gradually, by adding days to February for a couple of years in the early 18th century, which is where your 'Scandinavian calender' comment comes from, I assume.
