So, you do not know of any Scriptures that gives Pastors authority of the church.
Heres one.......
1 Peter 5:2–3 (ESV)
"shepherd the flock of God that is among you, exercising oversight, not under compulsion, but willingly, as God would have you; not for shameful gain, but eagerly; not domineering over those in your charge, but being examples to the flock."
Heres' another...
Titus 1:9 (ESV)
"He must hold firm to the trustworthy word as taught, so that he may be able to give instruction in sound doctrine and also to rebuke those who contradict it
Then there is Hebrews 13:17...
" Obey them that have the rule over you, and submit yourselves: for they watch for your souls, as they that must give account, that they may do it with joy, and not with grief: for that [is] unprofitable for you."
1 Thess. 5:12
"And we beseech you, brethren, to know them which labour among you, and are over you in the Lord, and admonish you;".
Acts 20:28........
"Take heed therefore unto yourselves, and to all the flock, over the which the Holy Ghost hath made you overseers, to feed the church of God, which he hath purchased with his own blood."
If you need more, let me know.