Take a big, deep breath.
Then let it out,..and tell yourself only you, control you, from here on out.
Here is what I would do in your situation. Stop hating him.
Obviously he is dealing with some type of mental issue. He is unstable.
Hating him, is just going to exhaust your energy.
If you truly want to leave him, you need to start keeping evidence of his whereabouts. Chat logs, pics of him with her, whatever you need, so you have evidence of his 'stepping-out'.
The more evidence you have of his long-term secondary relationship, the less of a leg he has to stand on, trying to label you the 'whore' and have the kids taken away.
- You can`t have to much evidence.
- You need to tuck money away, if you haven`t already, and build yourself a nest egg. Sell valuables, whatever you need to do. Get money that can be used for leaving.
Start planning, and getting yourself prepared for life 'after him'. Do not let yourself build, and build until you blow, and leave without a plan. Building a plan will help you stay calm a little while longer, and handle things smartly.
Once you are long, and away from him, you need to get involved in some therapy for yourself and the kids. You have some responsibility in the man you picked. Obviously that is a different topic for a different day, but it is something that you need to evaluate and learn from before diving into the world of poly again.