The apostasy did not begin while apostles were still on earth. That's ridiculous.
If the early church did not need temples then we don't need them today.
Act 20
29 For I know this, that after my departing shall grievous wolves enter in among you, not sparing the flock.
30 Also of your own selves shall men arise, speaking perverse things, to draw away disciples after them.
2 Peter 2
1 But there were false prophets also among the people, even as there shall be false teachers among you, who privily shall bring in damnable heresies, even denying the Lord that bought them, and bring upon themselves swift destruction.
2 And many shall follow their pernicious ways; by reason of whom the way of truth shall be evil spoken of.
3 And through covetousness shall they with feigned words make merchandise of you: whose judgment now of a long time lingereth not, and their damnation slumbereth not.
2Thess 2
"Let no man deceive you by any means: for that day shall not come, except there come a falling away first, and that man of sin be revealed, the son of perdition;.....For the mystery of iniquity doth already work"
1Tim 1
15 This thou knowest, that all they which are in Asia be turned away from me; of whom are Phygellus and Hermogenes.
1John 2
18 Little children, it is the last time: and as ye have heard that antichrist shall come, even now are there many antichrists; whereby we know that it is the last time.
19 They went out from us, but they were not of us;
Jude
4 For there are certain men crept in unawares, who were before of old ordained to this condemnation, ungodly men, turning the grace of our God into lasciviousness, and denying the only Lord God, and our Lord Jesus Christ.
You either believe the Bible or you don't.
*The early Church did have temples maybe not as big and as splendid as we have today, maybe it was just a tent or up on the Mountain of God but they had them.