Think of it this way,
The Orthodox Church has a "checks and balances" system, in a way, similar to the United States Government...
We have Laity, Clergy and Monastics.
Each group is expected to balance the others out. When one group gets out of line, it's the job of the others to reel them back in.
Even the Clergy have a "checks and balances" system built in...
You have:
(Minor Orders)
Acolyte
Cantor
Reader
Subdeacon
(Major Orders)
Deacons
Priests
Bishops
The Priests are organized into various titles, though while some are administrative, they are mostly titles of honor...
Then, amongst the Bishops, you have an additional "checks and balances" system...
Titular Bishop
Auxiliary Bishop
Vicar
Archbishop/Metropolitan
Patriarch
Each group has a unique duty to serve and unique jobs, and each group serves to balance the others out, and to correct them.
We do have a sort of "chain of command", but it isn't always absolute, and in case of heresy/apostasy, doesn't have to be adhered to.
Again, we have the example of the Robber Council of Florence, where the Laity and Monastics served as representatives of the Orthodox faith and corrected the Bishops.
You also have the example of the laity adopting Arianism almost en-masse and then being corrected by the monastics and the clergy.
This "checks and balances" can also be seen when Priests or Bishops are ordained. The people (laity), the Priests, the Bishops and the Monastics all together have to pronounce either "Αξιός" or "Ανάξιος". That is, they have to proclaim the candidate to either be worthy, or to be unworthy. If anaxios is proclaimed, then the ordination CANNOT proceed. If it indeed does, then it is uncanonical and is against the wishes of not just the people but is against the will of the Holy Spirit.
In this manner, you don't have a domination by Bishops, nor is the Church subject to some sort of "mob rule" or some form of "congregationalism".
Only in it's whole, can the Church be called "infallible". The whole of the Church also includes the entirety of History, and not just in time, but in eternity as well. So it's far beyond just the 250 million Orthodox living today.
This is why we cannot budge on our faith. We are representing not just ourselves, but the Saints in heaven, all the Church Fathers, all the Disciples and Apostles, all the Old Testament Prophets and Patriarchs. All the righteous faithful from this era to the beginning of time.
It isn't the "majority" that holds the truth, but it is those that hold the truth and the faith that hold the truth. Even if the majority apostatize, the few that remain faithful are still Orthodox and are still a part of the Church. This is why we can say that St. Mark of Ephesus, even though he was 1 bishop amidst 30, that he alone held the entirety of the faith handed down to us by the Apostles.
The faith has to remain unaltered and unchanged from the time of the Apostles. By this adherence to the faith, and through the works that come about by that faith, we will be judged by the Lord our God.