Here's
in the third century:
Originally Posted by St. Cyprian, Ep. 51 Moreover, Cornelius was made bishop by the judgment of God and of His Christ, by the testimony of almost all the clergy, by the suffrage of the people who were then present, and by the assembly of ancient priests and good men, when no one had been made so before him, when the place of Fabian, that is, when the place of Peter and the degree of the sacerdotal throne was vacant; which being occupied by the will of God, and established by the consent of all of us, whosoever now wishes to become a bishop, must needs be made from without; and he cannot have the ordination of the Church who does not hold the unity of the Church.
You look for doctrines being explicitly defined in Scripture or in the Early Church Fathers. They didn't define every point of doctrine explicitly. That doesn't mean it wasn't there, it just wasn't understood as fully as later.
Yet there is plenty of evidence that they acted as though they believed it was true, even if they didn't sit down and write out an argument for the Primacy of Peter. This is true as far back as the first and second centuries when Rome clearly exercised authority over the other Sees. Pope St. Leo the Great made the argument when the primacy was challenged by Constantinople but prior to this time there had been no real need to discuss it in depth, in the sort of structured logical that you might desire, at least not in what has come down to us today.