I am at a loss as to why you want to continue to be married to someone who might say he loves you and yet goes out of his way to show you otherwise.
He agreed to alot of things regarding open marriage and didn't stick with those rules either. So what makes you think he will "obey" any rules when it comes to losing you and his kids and possessions? (divorce)
You owe it to your children to make sure they have security and putting your head in the sand thinking he will stay and life will be wonderful or divorce will be easy is just not facing reality. Your marriage does not sound as good as you might think it is if this is his behavior.
I have an ex husband that "agreed" to be cordial in case of divorce. We didn't even step foot in a court room. BUT as soon as those papers were final he did everything, and continues to do everything he can to be hurtful.
I of course don't know your husband, you do. He has shown you that he can be hurtful (by asking you to go to his wedding), he has shown you can be manipulative (he can see you even if a divorce were to happen due to the kids, he will just walk away from both, so that you both call and swoon all over him). You had to MAKE him stay (though not sure how or why).
Why would you not let him go? if even for a little bit of time. Just to spend some time thinking about yourself. Why are you letting him hurt you? Why are you not demanding him to leave if he is only going to treat you badly? You might need a little time for some personal introspection.