You are mistaken.
2 Col 2:16
Therefore do not let anyone judge you by what you eat or drink, or with regard to a religious festival, a New Moon celebration or a Sabbath day.
Acts 15
5Then some of the believers who belonged to the party of the Pharisees stood up and said, “The Gentiles must be circumcised and required to keep the law of Moses.”
6The apostles and elders met to consider this question.
7After much discussion, Peter got up and addressed them: “Brothers, you know that some time ago God made a choice among you that the Gentiles might hear from my lips the message of the gospel and believe.
8God, who knows the heart, showed that he accepted them by giving the Holy Spirit to them, just as he did to us.
9He did not discriminate between us and them, for he purified their hearts by faith.
10Now then, why do you try to test God by putting on the necks of Gentiles a yoke that neither we nor our ancestors have been able to bear?
11No! We believe it is through the grace of our Lord Jesus that we are saved, just as they are.”
The bottom line is that we must obey God rather than man, so if God said to keep the Sabbath and you think man said not to keep the Sabbath, then you should obey God and disregard what man said. According Deuteronomy 4:2, it is a sin to add to or subtract from God's law, so if Paul had told anyone not to obey the Sabbath, then he would have been sinning. Furthermore, according to Deuteronomy 13:4-6, if anyone teaches against obeying any of God's commands, then they are false prophet, even if they perform signs and wonders, so if Paul taught anyone against keeping the Sabbath, then he was a false prophet and we should disregard what he said. However, if we are careful not to misinterpret something that is against obeying man as being against obeying God, then we will see that Paul never taught anyone against obeying God.
My claim was that for roughly 7-15 years after Christ's ascension up until in the inclusion of Gentiles in Acts 10 all Christians were Torah observant Jews, so quoting verses that take about what happened after Acts 10 does not show my claim to be mistaken. Stephen was falsely accused of teaching against the law (Acts 6:13) and Paul was also falsely accused of teaching against the law and took steps at the direction of James to show that he continued to live in accordance with the law (Acts 21:20-24), so if no one was teaching against Torah observance, then that is what they continued to do. If Jewish Christians hadn't continued to be Torah observant, then the problem raised in Acts 15:1 would never have been an issue. However, the problem raised in Acts 15:1 wasn't whether Gentiles should obey God, but whether they needed become circumcised and obey man-made customs in order to be saved, which they rejected.
The Jerusalem Council had no authority to tell Gentiles not to obey God's law, nor did they do so. A rabbi's yoke was the way that he taught to obey God's law, or in other words, his traditions. The yoke that they were referring to in Acts 15:10 was the yoke of the Pharisees, which Jesus was also critical of (Matthew 23:3-4), where they buried people under the weight of their many man-made traditions. In Mark 7:6-13, Jesus was also critical of the Pharisees for setting aside the commands of God to follow their traditions. In Matthew 11:28-30, Jesus wanted people to find rest for their souls by following his yoke or his way of keeping the law, which was a reference to Jeremiah 6:16-19, where God's law is equated with the good way where people will find rest for their souls:
Jeremiah 6:16-19 Thus says the Lord: “Stand by the roads, and look, and ask for the ancient paths, where the good way is; and walk in it, and find rest for your souls. But they said, ‘We will not walk in it.’ 17 I set watchmen over you, saying, ‘Pay attention to the sound of the trumpet!’ But they said, ‘We will not pay attention.’ 18 Therefore hear, O nations, and know, O congregation, what will happen to them. 19 Hear, O earth; behold, I am bringing disaster upon this people, the fruit of their devices, because they have not paid attention to my words; and as for my law, they have rejected it.
So pay attention because if they had been saying that God's law was a burden no one could bear, then they would have in disagreement with God, who said that his law was not too difficult for them (Deuteronomy 30:11-14). Furthermore, 1 John 5:3 says that God's commands are not burdensome. God did not give the law to be a heavy legalistic burden that the Pharisees perverted it into, but rather it was given for our own good (Deuteronomy 10:13) and meant to be received by faith as a delight and a divine privilege (Romans 7:22, Psalms 1:1-2).
In regard to Colossians 2:16, it can be ambiguous just from that verse whether they were being judged for keeping God's holy days or for not keeping them, so we need to look at the surrounding context to determine which is which, primarily at what the views of the people judging them were.
Colossians 2:8 See to it that no one takes you captive by philosophy and empty deceit, according to human tradition, according to the elemental spirits[a] of the world, and not according to Christ.
Paul would never describe God's holy, righteous, and good law as empty deceit or according to human tradition. Christ kept God's law perfectly so it is according to Christ and say that it is not would be pitting the Son against the Father. Paul went into more details about what these elemental spirits are later in the chapter:
Colossians 2:20-23 If with Christ you died to the elemental spirits of the world, why, as if you were still alive in the world, do you submit to regulations— 21 “Do not handle, Do not taste, Do not touch” 22 (referring to things that all perish as they are used)—according to human precepts and teachings? 23 These have indeed an appearance of wisdom in promoting self-made religion and asceticism and severity to the body, but they are of no value in stopping the indulgence of the flesh.
So the Colossians were keeping God's holy days in accordance with His commands and were being judged for keeping them by those who were teaching human precepts and traditions, self-made religion, asceticism, and severity to the body. Paul was telling them to obey God rather than man and encouraging them not to let anyone keep them from obeying God. So again, we must be careful not to mistake something that is against obeying man-made traditions as being against obeying God.
