My advice may sound a bit archaic but my beliefs and views are archaic in nature so my advice will always sound a little "out of date". That being said, I think you should stick with the woman you made a child with. God loved your union with this woman so much that he blessed the two of you with a child, his greatest gift. I don't think you should take that lightly. IMO God chose for you to be with this woman. While you never got married to her you had a sexual union with her before you were married. IMO, if you were man enough to make a baby with her you should be man enough to either break up with her and pay child support and fight for partial custody or be man enough to marry her and stick by with her. I'm confused by your statement of "it always ends of sinning when we try to get back together." do you mean that you always have sex with her? Why would that be a reason to end the relationship? People succumb to the temptations of sex before marriage all the time. My wife and I did for a year before we got married. Yes, its one of my biggest regrets but, I also loved her enough to have sex with her and make that commitment and union with her, I should be man enough to marry her. Your situation is even more different than mine, God blessed you with a child from this woman. Isn't that reason enough to stick together? I honestly don't understand.
I mean, you're not constantly cheating on each other are you? Imo that should be the ONLY reason to split ties with this woman. It could be what you meant by "we keep on sinning" but, if you're waiting for the perfect relationship with the perfect girl whom everything is perfect with, you're living in fantasy land.
Couples argue and stuff like this happens. All the time. My wife and I love each other and are committed to each other but, we do fight. Almost constantly. Still doesn't change the fact that got chose for us to be together and that God wants our marriage to stand until the day I die. IMO sexual union is like that. It's not "cheap" or "hey lets have a casual encounter and never see each other anymore." But, idk... Idk what to tell you. I gave you the advice you didn't want to hear so... *shrug*