I personally agree with you, the thought being that God looks upon the inner heart of man, and not the outer appearance. Also as you point out, obeying the Lord in remembrance of Him through partaking of the Lord's Supper is something we are also to do as believers in Christ. So it can also be said that abstaining from the Lord's supper as a follower of Christ is actually a neglecting of the person's walk with Him even though he or she has not yet been baptized, and therefore should be allowed.
(As an aside), Interestingly enough I remember in some Regular Baptist churches I attended that the "ceremony" (for the lack of a better word) of the Lord's Supper usually entailed simply a reading of 1Corinthians 11:17-34, a moment of introspection, a blessing for the bread, and a blessing for the grape juice, being careful to partake all at the same time in order to be orderly.
I think the thing was back in the 1800s a lot of churches used to actually "police" their "tables" to ensure nobody just walked in to a service, and seeing that people were performing this "strange practice", ate the cracker and the juice without any relationship with Jesus whatsoever.
(And a second aside), I visited a Catholic Church a few times back in the '90s, and they also ask people not to partake in the Eucharist if they are not baptized and confirmed Catholics. So the practice is not completely unheard of.
I just wonder if "being responsible for people for not eating and drinking a curse of death upon themselves", is something that should be a concern for us while we are doing something that Jesus wants us to do?