You have some decisions to make before you proceed with any action. You need to pray and ask God to guide you. Without question, God is on your side. You don't deserve this treatment, it is betrayal and beyond explanation. Exclude her and focus on him. This is all him. He's married, has a responsibility to honor his marriage, his vows, his children, you and his God, and all who love you guys as a couple. You need to decide if you want this marriage. If you do, you should be prepared for the long road to restoration. I personally think that as long as he hasn't blatantly flaunted this relationship in your face that there's room for reconciliation. God is a God of reconciliation and restoration. That's the entire message of the gospel. It's always one person sacrificing for the good of many. You will bear, and have been bearing, the pain of your relationship. If he's a good father, a good friend, in spite of being a sucky husband, then pray for him, and insist on counseling and transparency.
If you do decide to move forward, you should ask for access to all of his social media accounts, email accounts, phones, tablets, pc, and laptop. He needs your help to break that connection and recover from that addiction to the pheromones and dopamine. Transparency and accountability is the best way to hold him accountable. There should be days, unannounced that you switch phones for the day when he goes to work. This is a lot of work, but I'll assure you, your marriage is worth it if you decide to keep it, and I would encourage you to, especially since there doesn't appear to be any violence or abuse of any kind. If he doesn't comply, you may want to pray about whether it's in your best interest to stay in the marriage. If he does, the next step would be to have him type out apology letters to the young lady, or ladies (based on your discoveries when viewing his social media connections) voicing his commitment to you, his marriage, children, and God and advising her to never make contact with him again, etc. You should approve these and send them yourself. You should close all of his social media accounts and create one for both you guys and refriend those of your choice that are on his old accounts. All his social media connections should be approved by you; you should have complete control/access to every part of his life and vice/versa. I know this sounds extreme, but nothing is as extreme and permanent as divorce. This will actually remove the walls and bring you guys closer than you've ever been. You'll grow past this and into a new life of open communication and deep intimate connections.
You are worth more than he's giving you. You've given your life to him and he owes his life to you and both of you owe the best shot you have to your kids and all who love you. For some reason people don't consider the pain of people who love them, the families that idolize them, the children's pain of dealing with being a statistic as they talk to their friends who have complete households. But you are worth it...some who love you may say you deserve better, and they may be right, but what you don't deserve is the pain of leaving your husband; the pain of divorce, division of marital property, custody fights, visitation arrangements, arguments over child support and/or spousal support, division of assets, etc.
There are demons that come alive during divorce that sew discord that could damage your friendship forever. One of the reasons God hates it. These demons specialize at making the most of those vulnerable in divorce, especially the kids. It's unpredictable, and it takes a lot of prayer and preparation to have a peaceful divorce, these are very rare. Believe me, you may deserve better than him, but you also deserve better than the experience of divorce; so does your children.
Love your husband, forgive him (either way, whether you keep him or free him, forgive him) if you don't forgive him you will be the one suffering, not him. Forgiving him frees you. Be completely open with him and require the same from him. I'll keep you guys in my prayers, especially you. You have a lot of decisions to make. Your decisions should be focusing on permanence...getting things set for a permanent future. Accept nothing that you can't live with forever...accept nothing that distroyed your marriage, from you or him. Take responsibility for your role, admit it, repent of it, and move on. God's richest blessings be upon you and your family.