So, do you think Heaven is an actual, physical place somewhere? Or more like a spiritual dimension?
Neither.
"Heaven" in these sorts of contexts is euphemistic language for the utmost, transcendent where God "dwells".
Where Jesus says "Kingdom of Heaven" in Matthew's Gospel, the other Gospels have Jesus saying "Kingdom of God"; "Heaven" being a euphemism for God.
It's to look up at the heavens--where the clouds, the stars, the planets, all that "dwell"--and to say "beyond all of that, beyond the highest of everything God is. Not that God dwells in some dimension or location or plane located beyond the material universe; that's not the point. The point is to address the vastness, the beyondness of God and thus "Heaven" or "the highest heaven" is to speak of God's own transcendence; not as though He were some sort of
thing that exists in another dimension or what not.
God is both transcendent and immanent. Beyond all things and pervading all things. He is everywhere at all times, never lacking.
Jesus is at the right hand of God, that speaks to His power and authority as the Risen Christ seated to reign as Lord and King over all things; not of a physical (or even "spiritual") location.
As ebia noted, the whole physical-spiritual dichotomy is basically Gnostic dualism which historic, orthodox Christianity rejects. Which is why one should answer neither to your question, and instead understand what "Heaven" language means biblically, and that to understand what it means to say Jesus is seated at the right hand of God.
-CryptoLutheran