Nope ---
- Had the star been a celestial object --- everyone would have seen it.
- The star initially led them to Jerusalem, not Bethlehem --- it was Herod that sent them to Bethlehem with orders to search-and-locate.
- The star then reappears and guides them to an exact address.
What celestial object can lead someone to an exact address, yet not be seen by anyone but a few chosen men?
1. Everyone may have seen it. But only those trained to read the stars would divine its meaning.
2. The text does not say the star led them to Jerusalem. The wise men explain they have come to Jerusalem to find the king of the Jews because "we observed his star at its rising." That, as we have deduced, was two years earlier. Nothing in the text requires that they saw it every night or that it travelled before them like a guiding beacon. That is a Sunday School Christmas Pageant convention (which also erroneously has the wise men arriving at the stable with the shepherds.)
As students of the stars, the wise men only needed to know the position of the star (possibly in Pisces) to know its meaning. They didn't need it to lead them to Jerusalem. They would go there because that is where they expected to find a king of the Jews.
3. Yes, the star, whatever it is, reappears and stops over Bethlehem, confirming the scripture which the priest had quoted to Herod.
IIRC Bethlehem is somewhat to the south and west of Jerusalem, so now the star is setting. Note that if it was not too high in the sky, it could be seen from Bethlehem while being invisible from Jerusalem. So, it would "reappear" as the wise men approached Bethlehem.
Some have suggested that the star was a conjunction of Jupiter and Saturn in Pisces. This would make sense, for a conjunction can be repeated several times as a planet moves past a conjunction and then retrogrades to meet it on its reverse path again. A third conjunction would occur as the faster planet resumed its forward motion again. The link below discusses this hypothesis.
http://www.unmuseum.org/bstar.htm
Of course, that is only one possibility, and I don't swear by it. But the reasons you gave are not reasons to say it could not be a celestial body.