The cons are that we are no longer in a society where holiness is valued. The chances are slim that merely removing oneself from a church, without also removing oneself from society, will be an effective path to holiness.
It seems to me that we have polar opposites here.
Hermits removed themselves from the world for the sake of holiness, but from all I have read did not seem to reject the Church itself.
The approach I thought we were discussing here usually involves removing oneself from the Church, but remaining in the world.
I agree that the chances of this effecting holiness are "slim" (or much worse, in fact). But it really seems the opposite of what ought to be, if we see any wisdom in the desert fathers/hermits/etc.
Throwing off the guidance and accountability of the Church and the fellowship of the faithful and all the associated benefits - in an environment of the world that is is largely opposed to holiness - just seems to me like a dangerous path.
I know what God led me to do, which is supported Scripturally and taught by the Church, and I went through a lot of difficulty in order to be obedient to that, to my very great benefit, for which I am thankful.
As I said before, I feel badly for those who truly have no choice, but for someone to deliberately choose such long-term, seems likely to be a matter of rebellion, and most unwise.