Which has absolutely nothing to do with the simple truth that Psalm 122 speaks of The Jerusalem which is on earth, the city of the great King.
I think Scripture clearly speaks of two Jerusalems. There was the natural earthly Jerusalem that represented the Old Covenant system. Then there was the spiritual heavenly Jerusalem from above, the New Jerusalem, which represented the New Covenant bride of Christ.
Paul used an allegory to explain this concept in Galatians chapter four. He associated Hagar, Ishmael, Mt. Sinai, and slavery with the Old Covenant and his then present city of Jerusalem. He then associated Sarah, Isaac, and freedom with the Jerusalem from above, the New Jerusalem. This would become the New Covenant system.
He told the believers that they were children of promise which meant they were free from the bondage to the present system and were members by faith of the new city from above. He then reminded them that just as at that time those of the flesh persecuted those of the spirit, so it was in their day. The unbelievers of Old Covenant Israel were persecuting those who were coming to Messiah in the last days.
Then Paul quoted Genesis 21:10 ..."Drive out this maid and her son, for the son of this maid shall not be an heir with my son Isaac."
In this allegory the maid, the bondwoman, represented the old city of Jerusalem, the Law, the Old Covenant system. She would be cast out leaving the promised New Jerusalem, "the mother of us all," as the sole representative of God's New Covenant people. This event was consistently described in terms of imminence throughout the New Testament.
The old city was judged and destroyed in AD 70 as the new city was fully revealed.
I believe John described it clearly in Rev. 21:2-3
"And I saw the holy city, new Jerusalem, coming down out of heaven from God, made ready as a bride adorned for her husband. And I heard a loud voice from the throne, saying, "Behold, the tabernacle of God is among men, and He will dwell among them, and they shall be His people, and God Himself will be among them"