Galatians 4:9,10 does mean the feast days, but it also means the yearly sabbaths (7-year and jubilee), as well as the new moon sabbaths and the weekly sabbaths. There are many verses in the old testament that use the phrase "days, months, seasons, and years" or variations thereof and these all refer to the weekly, as well as what you would consider the ceremonial sabbaths. See 1 Chronicles 23:31, 2 Chronicles 8:13, 2 Chronicles 31:3, Nehemiah 10:33, Isaiah 1:13,14, Ezekial 45:17, Hosea 2:11.
That fourth commandment says more than "remember the sabbath day". It requires you not to work. It also requires you not to make others work. When I was an SDA and went to church every week, I and every other person in our church broke that commandment every week when we showered, turned our lights on, ran the air conditioning, listened to the pastor over the loud speakers, reheated the sabbath dinners that had been prepared on friday, and everything else that required the folks down at the electric company, gas company, phone company to work so that we might enjoy our sabbath rest.
I agree with you. Paul is in no way saying that we have a license to sin. But I also wonder why, if the sabbath was so important, the council at Jerusalem in Acts 15 didn't include it with the other stipulations for the gentile christians.