That's quite a history between you too. If someone asked me to quickly summarize what I just read, it would go like this:
He does what he wants to, when he wants to, how he wants to. You put up with it, hoping hour by hour, day by day that he will change. He pressures and threatens you (custody threat) into marrying you, then does whatever he wants. He cheats on you, lies about it, then when you catch him he tries to excuse it by calling it poly, and his new spiritual awareness. You get his attention only when you REALLY get mad at him. He accuses you of being possessive and calls himself poly, when all you're really doing is asking him to be a responsible husband and father. He doesn't want to be responsible. He doesn't want to be a husband. He doesn't want to be a father. In fact, after he lies and cheats, he works really hard to convince you that you are the one doing all the destructive things by calling you possessive. He tries his best to convince you that you are the destructive one, so you'll continue to put up with the abusive things he does. Sometimes this strategy of his works.
Poly requires honesty. Poly is way to fancy for where you too are right now. You need to take a close look at all the abusive stuff he does and why you put up with it. He does what he wants, then lies about it when he thinks the truth will make you mad. He is a baby. No one is requiring that he grow up. Why do you put up with all this abuse? Where is your self esteem?
My words are probably very unpleasant for you to read, but that is what I see in the story you told. The only way you're going to find the strength to improve your life is to look within yourself and find the self esteem that is there somewhere.
I wish you the very best. I hope you find the courage within yourself to give yourself the very best.