There is this part in Irenaeus letter to the Magnesians that really caught my attention to this debate. Ireneaus wrote letters on his way to his martyrdom in Rome, roughly around 107AD (middle of Trajan's reign 98-117, though some think under Hadrian's reign 117-138). This means that Irenaeus is really around before any formal Catholic institutions are around, him being alive in the first Century and very early Second.
In chapters 8 through 10 he discusses Christianity's relationship with Judaism. His words are as follows:
8: Do not be deceived by strange doctrines or antiquated myths since they are worthless. For if we continue to live in accordance with Judaism, we admit that we have not received grace. For the most godly prophets lived in accordance with Christ Jesus. This is why they were persecuted, being inspired as they were by his grace in order that those who are disobedient might be fully convinced that there is one God who revealed himself through Jesus Christ his Son who is his Word that come forth from silence, who in every respect pleased the one who sent him.
9: If then, those who had lived according to the ancient practices came to the newness of hope, no longer keeping the sabbath but living in accordance with the Lord's day, on which our life also arose through him and his death (which some deny) the mystery through which we came to believe, and because of which we patiently endure, in order that we may be found to be disciples of Jesus Christ, our only teacher. How can we possibly live without him whom even the prophets, who were his disciples in the Spirit were expecting as their teacher? This is why the one for whom they rightly waited raised them from the dead when he came.
10:Therefore, let us not be unaware of his goodness. For is he were to imitate the way we act, we are lost. Therefore, having become his disciples, let us learn to live in accordance with Christianity. For whoever is called by any other name than this one does not belong to God. Throw out, therefore, the bad yeast which has become stale and sour, and reach for the new yeast which is Jesus Christ. Be salted with him so that none of your become rotten, for by your odor you will be convicted. It is utterly absurd to profess Jesus Christ and to practice Judaism. For Christianity did not believe in Judaism but Judaism in Christianity, in which every tongue believed and was brought together to God.
This passage is a longer quote I know, and there are many tangential issues within this long passage, but the point is, to Irenaeus, that keeping the Saturday as Sabbath was the opposite of what a Christian should do. Instead, we should keep the Lord's day (Sunday) because Christ rose, and Christ is the hope of our Salvation, not the Jewish law. His words are pretty harsh, but they seem effective.
Not that we should make a doctrine off of Irenaeus, but it is interesting to ponder, and think about.
For me, personally, I like the letter of the Jerusalem Council. This is the Apostles deciding what is fit for Gentiles to keep. (Hint, it doesn't mention Sabbath at all)
Acts 15:23-29
"They wrote this letter by them:
The Apostles, the Elders, and the Brethren, to the Brethren who are of the Gentiles in Antioch, Syria, and Cilicia, Greetings:
Since we have heard that some who went out from us have troubled you with words, unsettling yours souls, saying "You must be circumcised and keep the law," to whom we have no such commandment, it seemed good to us being assembled with one accord to send chosen men to you with our beloved Barnabas and Paul, men who have risked their lives for the name of our Lord Jesus Christ. We have therefore sent Judas and Silas who will also report the same things by word of mouth, for it seemed good to the Holy Spirit and to us, to lay upon you no greater burden than these necessary things: that you abstain from things offered to idols, from blood, from things strangled, and from sexual immorality. If you keep yourselves from these, you will do well.
Farewell."
The highlight of course is: "to lay upon you no greater burden than these necessary things" which the list does not include strict Sabbath laws. The gentiles never had a practice like the Jews in respect to the 7th Day, and it appears that the Apostles did not find the need to request that they keep it. Therefore, for me, it is not a burden that any gentile has to worry about.