Much though I don't want to say this. . .you need to break up with him. God says we are not to be married to unbelievers.
"Do not be yoked together with unbelievers. For what do righteousness and wickedness have in common? Or what fellowship can light have with darkness? What harmony is there between Christ and Belial? Or what does a believer have in common with an unbeliever?" 2 Corinthians 7:14-15
This verse tells us that we shouldn't be "yoked", or put together with, an unbeliever, and tells us why.
But some people say, "I might be able to lead them to Christ!" or, "They're not a bad person."
The Bible says, "“I have the right to do anything,” you say—but not everything is beneficial. “I have the right to do anything”—but I will not be mastered by anything. You say, “Food for the stomach and the stomach for food, and God will destroy them both.” The body, however, is not meant for sexual immorality but for the Lord, and the Lord for the body. By his power God raised the Lord from the dead, and he will raise us also. Do you not know that your bodies are members of Christ himself? Shall I then take the members of Christ and unite them with a prostitute? Never! Do you not know that he who unites himself with a prostitute is one with her in body? For it is said, “The two will become one flesh.”" 1 Corinthians 6:12-16
As a Christian, you have been washed clean from sin, and forgiven. These verses talk about how we might be able to do something, but it's not good for us. You might be able to drink 14 Red Bulls in a day, but that's definitely bad for you! Here, these verses are specifically talking about sexual immorality, but it also talks about how you are part of Christ, and how we shouldn't be united with that which is not of Christ.
I really don't want to have to tell you that you need to exit this relationship, and I understand your pain. I was once in love with an amazing guy, who was kind, sweet, smart, understanding, and so much more, but he didn't believe. When I began to get closer to God, I realized that God. . .doesn't want us to be married to those who don't believe. It's a hard thing, and I cried a lot over it. Being in close friendship with someone who doesn't share your beliefs will hurt your relationship with God, and that makes it either God or the person. I chose God, and it was REALLY hard -- but it's been worth it. I gave up my life to God, and told Him I would be happy with whatever path He took my life down. Almost a year later, I started dating the man I'm fairly certain I will marry, and he is amazing in so many ways, and he knows and loves God! God knows best, and He WILL provide for you!
I'll be praying!