The problem with that statement is "the Lord's Day" is not Sunday. The term was used in the Prophets ("The Day of the Lord") in dealing with end time events and thus when John, in Revelation, uses the term... he is not denoting "Sunday," he is drawing on a phrase understood by his contemporaries to be dealing with end time events. And so, there he sat, in the Spirit viewing the end time events and recording them. After the first century and once the weight shifted from the faith being more or less a sect of Judaism to distinguishing itself as a new religion, Greek philosophy entered and thus so too did the spiritualization of many things including the day of the resurrection which CULTURALLY was called, "The Lord's Day." So, he raised on Sunday and non-Jewish believers began to call that day the Lord's Day... and on the authority of the church and NOT Scripture... they shifted the Sabbath from Saturday to Sunday.
"It is well to remind the Presbyterians, Baptists, Methodists, and all other Christians, that the Bible does not support them anywhere in their observance of Sunday. Sunday is an institution of the Roman Catholic Church, and those who observe the day observe a commandment of the Catholic Church." Priest Brady, in an address reported in the Elizabeth, NJ News on March 18, 1903
"Protestants ... accept Sunday rather than Saturday as the day for public worship after the Catholic Church made the change... But the Protestant mind does not seem to realize that ... in observing Sunday, they are accepting the authority of the spokesman for the Church, the pope." Our Sunday Visitor, February 5, 1950
"The [Roman Catholic] Church changed the observance of the Sabbath to Sunday by right of the divine, infallible authority given to her by her founder, Jesus Christ. The Protestant, claiming the Bible to be the only guide of faith, has no warrant for observing Sunday. In this matter the Seventh-day Adventist is the only consistent Protestant." The Catholic Universe Bulletin, August 14, 1942, p. 4
So here are a few of many claims I can share proving that even the RCC acknowledges the change was their doing and not Scripture. And when did they do this?
The council of Laodicea states in cannon 29:
"Christians must not judaize by resting on the Sabbath, but must work on that day, rather honouring the Lord's Day; and, if they can, resting then as Christians. But if any shall be found to be judaizers, let them be anathema from Christ."
It took a decree AND the threat of ex-communication, to shift the church away from the Sabbath. We were then born 1600 years later into a culture that saw Sunday as the Christian Sabbath and we simply don't even know the was a question to ask.
* As a side note... if they had to pass a decree, then clearly there were enough Christians still honoring the Sabbath that a decree was even necessary.