Look at this timeline...

If you remember, after the fall of the secular western Roman empire, the remaining eastern secular Roman emperor (who's also pretty much ran the eastern church as well)

I'm sorry, your "historical" recount is flimsy at best. The fact is, the church has NEVER been "comprised" of independent groups who just happened to agree with each other and "form a whole". The fact is, Jerusalem, Antioch, Alexandria, and Constantinople were all Roman cities! All within the Roman empire and all existing under the Roman umbrella. Additionally, the Roman empire was seated in two main places. They were Constantinolpe in the east, and the City of Rome in the West. And as you already should know, the government of the empire flowed from the City of Rome in the West.

Ironically it was the western part of the secular Roman empire which collapsed first and the church was laid bare to the invasion of the Teutonic peoples. If you remember, after the fall of the secular western Roman empire, the remaining eastern secular Roman emperor (who's also pretty much ran the eastern church as well) tried to assert doctrinal supremacy but was not afforded this position because the doctrinal authority of the Church in the City of Rome had already been established over the course of the preceding centuries. Some examples to refresh your memory are as follows:
Pope Victor (189-199) intervenes and settles a dispute in the Eastern church about the date of Easter.
Pope Stephen (254-257) defends the doctrinal primacy of Rome against Cyprian of North Africa .

However, it is really the council of Chalcedon in 451 where Papal primacy is established. Additionally, the years following the Council at Chalcedon indeed show a clear communion with and recognition of the authority which resided in the west. The Patriarchs and bishops all remained not simply in communion with each other, but subject to the authority which flowed from the Church in Rome. This authority goes right back to scripture in Acts chapter 15. We see the patriarchs and bishops debating about various laws and their application to Christianity. It is when Peter stands and speaks, that the arguments are settled. In verse 13, we see that the congregation "fell silent" after Peter's discourse. Only then, does James speak up and expand upon Peter's established point. After this, the congregation is "mysteriously" no longer debating but now in agreement.

The fact is, your idea of 5 patriarchs running the church in perfect harmony is rediculous on so many levels. Perhaps the most obvious being that every ecumenical council is (and has been) comprised of a world-wide congregation of Bishops, not simply 5 (or 4) Patriarchs!

That said, I need to state that the Catholic Church grants the Eastern church a special place of compassion and an attutude of sorrow and remorse for the 4th crusade. As you should know, the first crusade was actually initiated at the request of the eastern church who was being pummelled by the invading Muslims, and the Western church responded by sending the crusading armies to help the eastern church. The Armies initially helped defend the City of Constantinople and even reclaimed portions of the Holy Land from the Muslim invaders. However, the pillage and sack of Constantinople during the 4th Crusade (some years later) was a tradgedy which the western church remains to this day unable to make up for and the eastern churches remain unwilling to forgive.

This tradgedy however, in no way negates or undermines the facts of history and the doctrinal authority which exist in the Roman Catholic Church to this day.