Tertullian, in The Demurrer Against the Heretics (A.D. 200), noted of Rome, How happy is that church . . . where Peter endured a passion like that of the Lord, where Paul was crowned in a death like Johns [referring to John the Baptist, both he and Paul being beheaded].
In the same book, Tertullian wrote that this is the way in which the apostolic churches transmit their lists: like the church of the Smyrnaeans, which records that Polycarp was placed there by John; like the church of the Romans, where Clement was ordained by Peter. This Clement, known as Clement of Rome, later would be the fourth Bishop.
In his Letter to the Romans (A.D. 110), Ignatius of Antioch remarked that he could not command the Roman Christians the way Peter and Paul once did, such a comment making sense only if Peter had been a leader, if not the leader, of the church in Rome.
Irenaeus, in Against Heresies (A.D. 190), said that Matthew wrote his Gospel while Peter and Paul were evangelizing in Rome and laying the foundation of the Church. A few lines later he notes that Linus was named as Peters successor, that is, the second pope, and that next in line were Anacletus (also known as Cletus), and then Clement of Rome.
St Irenaeus, "Against Heresies", chapter III, (190 AD)
"...the very ancient, and universally known Church founded and organized at Rome by the two most glorious apostles, Peter and Paul; as also [by pointing out] the faith preached to men, which comes down to our time by means of the successions of the bishops."
Clement of Alexandria wrote at the turn of the third century. A fragment of his work Sketches is preserved in Eusebius of Caesareas Ecclesiastical History, the first history of the Church. Clement wrote, When Peter preached the word publicly at Rome, and declared the gospel by the Spirit, many who were present requested that Mark, who had been for a long time his follower and who remembered his sayings, should write down what had been proclaimed.
Lactantius, in a treatise called The Death of the Persecutors, written around 318, noted that When Nero was already reigning (Nero reigned from 5468), Peter came to Rome, where, in virtue of the performance of certain miracles which he worked by that power of God which had been given to him, he converted many to righteousness and established a firm and steadfast temple to God.