The early Church is found both in scripture and outside it.
The early Church did not keep the Sabbath.
The only time it is recorded that Jesus broke bread was on the first day of the week.(Acts 24:30)
On the first day of the week when we gathered to break bread (Acts 20:7)
"On the first day of every week, each of you is to put something aside and store it up, as he may prosper, so that contributions need not be made when I come." (1Cor 16:2)
He says that because that is the day them met for worship, not Saturday.
"We keep the eighth day [Sunday] with joyfulness, the day also on which Jesus rose again from the dead"(Letter of Barnabas 15:68 [A.D. 74]).
When you come together on the Lords Day, break bread and give thanks, having confessed your sins, so that your sacrifice may be pure. (The Didache 1st Cent)
Both of the last two quotes are from apostolic times.
The early church continued to meet on Sunday:
"Those who were brought up in the ancient order of things [i.e. Jews] have come to the possession of a new hope, no longer observing the Sabbath, but living in the observance of the Lords day, on which also our life has sprung up again by him and by his death"(Ignatius of Antioch: Letter to the Magnesians 8 [A.D. 110]).
Note, this is less than 20 years after the death of the last apostle. Ingnatius of Antioch was reputed to be a disciple of the apostle John and a friend of Polycarp who was also a disciple of John.
"But Sunday is the day on which we all hold our common assembly, because it is the first day on which God, having wrought a change in the darkness and matter, made the world; and Jesus Christ our Saviour on the same day rose from the dead" (Justin Martyr: First Apology 67 [A.D. 155]).
It's important to use the Bible and the Bible alone to form our doctrines. All these men you quote as saying Sunday is the Lord's are found outside of Scripture. These men came to their conclusion based on there interpretation of the verses you quoted. But keep in mind, Satan had a plan long ago to change God's Law and time so at some point this was going to happen. With this in mind, let's take a closer look at the verses they used to prove a Sabbath change.
Before we start, it's also important that we understand two things. First, we can break bread seven days a week if we want because breaking bread is just a term used for fellowship but just because we fellowship does not mean we do away with the Sabbath. Second, the Bible is clear, the Sabbath, which is the seventh day, is in fact the Lord's day.
Leviticus 23:3 You will work for six days, but the seventh will be a day of complete rest, a day for the sacred assembly on which you do no work at all. Wherever you live, this is a Sabbath for LORD.
Now let's take a look at the term 'Lord's Day'.
The term, "The Lord's day" was taken from Revelation 1:10.
Revelation 1:10 I was in the Spirit on the Lord's Day, and I heard behind me a loud voice, as of a trumpet.
There is not one mention in this verse of Sunday or Saturday being the Lord's Day so saying Sunday is the Lord's from this verse is something that has to be added by man.
Now let's continue to the 'first day' texts.
The first one is found in Acts 20:7.
Acts 20:7 Now on the first day of the week, when the disciples came together to break bread, Paul, ready to depart the next day, spoke to them and continued his message until midnight.
Right away we can see the reason for the gathering was because Paul was leaving the next day, not because it was a worship service or a change of the Sabbath day.
Another important thing to understand in this verse is that the Sabbath begins at sunset on Friday and ends at sunset on Saturday. This gathering happened after the close of the Sabbath at sunset on Saturday, which would now make it the first day. To us it would still be Saturday because our days go from midnight to midnight but, to a Jew, the days go from sunset to sunset. Paul talked from sunset Saturday (making it the first day) until midnight. Then at sunrise, which would be Sunday morning, Paul left for his journey. He walked a long distance and then took a boat. If Sunday was now the day of worship or the Lord's Day Paul would have never journeyed all day, but instead, he would have been in church preaching on the Sabbath, like it was His custom.
Acts 17:2 Then Paul, as his custom was, went in to them, and for three Sabbaths reasoned with them from the Scriptures.
The next verse we'll look at is 1 Corinthians 16:2.
1 Corinthians 16:2 Upon the first day of the week let each one of you lay by him in store, as he may prosper, that no collections be made when I come.
Some people use this verse to say a church offering was taken on the first day so they must have been in church but a quick look at this verse we will see this is not what Paul was saying. Paul was going away but when he returned he was going to take some supplies to the saints in Jerusalem. The important words in this verse are the words, 'Lay by him in store'. People interpret these words to mean a church offering but this is not so. These words, 'Lay by him in store' are referring to getting things together from home such as water, food, livestock or whatever else the suffering saints in Jerusalem would need. Some Bible translations make it clear.
Darby Bible Translation
1 Corinthians 16:2 On the first of the week let each of you put by at home, laying up in whatever degree he may have prospered, that there may be no collections when I come.
Weymouth New Testament
1 Corinthians 16:2 On the first day of every week let each of you put on one side and store up at his home whatever gain has been granted to him; so that whenever I come, there may then be no collections going on.
International Standard Version
1 Corinthians 16:2 After the Sabbath ends, each of you should set aside and save something from your surplus in proportion to what you have, so that no collections will have to be made when I arrive.
Paul knew how getting all this stuff together from their homes and barns would be labor intensive and time consuming so he specifically tells them to do it on the first day, rather than, on the Sabbath day.
The next verse we'll look at is John 20:19.
John 20:19 Then, the same day at evening, being the first day of the week, when the doors were shut where the disciples were assembled, for fear of the Jews, Jesus came and stood in the midst, and said to them, "Peace be with you.''
Some people say the disciples gathered in this verse on the first day to worship God. They use this verse to say that the Sabbath was now changed from the seventh day to the first day but a quick look at the context will show it to be a delusion. The disciples were not gathering to worship God, they were hiding for fear of the Jews. Jesus had been crucified and they didn't know what was going to happen to them so they hid. Some people go as far as to say that they gathered in these verses to celebrate the resurrection of Jesus but this also is a delusion. The truth of the matter is, the disciples didn't even know Jesus had risen at this point this is why they were hiding for fear that the Jews would kill them too just like they did to Jesus.
So as you can see, none of these 'first day' texts say what people are making them say. The Sabbath is still the seventh day and we should honor it as the Lord commands.
One more piece of evidence that many people have never noticed. Before Paul's conversion, while on the road to Damascus, he told Jesus how he would go from synagogue to synagogue imprisonnig Christians, those who believed in Jesus.
Acts 22:19 Paul replied, 'Lord, they themselves know that from synagogue to synagogue I used to imprison and beat those who believed in you.
In other words, these early Christians were worshiping on the Sabbath in the synagogues when Paul pulled them out to imprison them.