What a stressful time this is for you. It's understandable that you would feel frustrated and helpless now that you have to legally defend yourself. I will certainly pray for you and your family.
In your OP, you mentioned something very telling:
When I was first divorced and they would act up I would always give in to them because I did not want them to be mad at me and not want to come visit me. I wanted them to love their dad.
This is a very common response from divorced dads. You feel guilt over the divorce, so you want to do whatever it takes to make them like you.
In the Old Testament, David did the same thing. He felt guilt and remorse over his adultery with Bathsheba, and his subsequent murder of Uriah. As you read on, you find that David did not discipline his sons as he should. In fact, he was a weak father. The consequences of this passive parenting was that his own son tried to take his kingdom away from him and kill him.
Likewise, it sounds like your children are also experiencing the consequences. When your primary motivation is to avoid their anger, as you stated, then discipline is lacking. The children become unruly and disrespectful.
You also said that you put your hands on your son first. You were aggressive with him, and he responded in kind. You say you cursed at him. And then the violence began.
You say that your son is a hot head, so I'm sure you knew how he would respond.
You also say that you were wrestling with him. What you describe is not wrestling, it is assault. On both sides.
The trouble is, he is a kid. You are an adult. It is the responsibility of the father to behave appropriately. As scripture tells us, in your anger, do not sin. You need to keep a level head and realize that you are dealing with an undisciplined child. You are responsible for the escalation.
I can understand how your ex feels. As a momma bear myself, I am fiercely protective of my children. She heard their account of what happened, and is concerned. That is understandable.
The very best thing you can do now is apologize to your children. They need to hear it from you. And you need to never lose your temper like that again. They need to be able to look at you as a role model of how they should behave. It is the responsibility of every parent to exercise self control, now matter how hard the kids punch our buttons. We are the example.
Going forward, I encourage you to behave with Christ-like humility. Forgive your children, and love on them. It may take a while for them to warm up to you, but keep loving them.
I think it is really important for them to see you repentant, not defensive. They will learn so much from it, and hopefully incorporate it into their own lives. But first, you need to realize that your behavior was wrong. The more defensive you are of it, the more distance you will put between yourself and your children, as well as between yourself and your Lord. Jesus said "Blessed are the peacemakers (Mat 5.9) This is your chance to become one.