For starters, unless he said "this is the devils sign" you are only assuming that this "symbol" has the same meaning to him. Last week a friend called and told me about his wife and her kindergarten class. There was a little girl in it that points with her middle finger. All the other kids mock her and some are offended, but the little girl was never told that our "culture" has demonized a perfectly good "God-created" finger. My point, he MIGHT not have viewed that symbol as you did. He probably did... but you don't know for sure unless you asked him.
Secondly, I think you placed yourself in this position. I don't say that to offend you at all... this is a result of our religious culture. The "Great Commission" is taken as a call to convert the world to Christianity and as a result, as soon as any new Christian can speak about their very recent life changing decision... we encourage them to go witness to their friends and family... more often then not only causing strife and division. Those that walked with Yeshua did so every day, night and day, for 3 years before they were really encouraged to go share. That brings me back to the Great Commission... which is not a call to convert at all... it is a call to TEACH. It says in passive participle form, "go" (literally, "as you are going") "make disciples" (students). We can't force anyone to learn... one must seek in order to find. Therefore, the "as you are going" is simply a call to walk the walk, reflect the Lord in all you say and do... and when God draws a man unto Himself (John 6:44) and He directs them into your path, then do the job, teach. Because at that point they want to learn.
Bringing somebody into your home and then trying to share Christ shows your love for Christ... I don't question that... but if that man isn't seeking, he isn't going to find. By extension... if he isn't asking questions he won't hear your answers. You would have been better to just make sure each word that you spoke and deed you performed reflected the one you serve so that his light shines brightly to the dark world around you.