I don't know what the others have said as I haven't read yet. I might be back to add more in a bit, but I wanted to let you know that life does not end after kids and house and marriage and over 30 (or whatever age seems old)... it keeps going and you will have a richness to your experiences that you don't right now at your age. Your age is amazing in that there is so much to do and so many options... later the options are still there, but the priorities have changed and some things lose their luster. It's hard to explain. The amazement need never have to leave though... its a choice to be amazed at lifes possibility.. I think quite often people are afraid of it and their own potential and decide to suppress their amazement or excitement. That is another story. sorry, getting off track
What I am trying to say with this is that you don't know what will happen. Just as you never thought this guy would EVER be in your life, he is. You don't know what will happen if you let him go; so that he can do what he thinks is his destiny. It sounds like he needs to hold on to what he thought was the best choice for him... by staying with him, I think, you are keeping him from the course he has set and a major course at that. He deserves to be free of a poly situation if his goal is to be monogamous and married with kids. The only one that can do that for him is you... by walking away.
By setting him free he may just come back to you. Maybe not soon, but at some point. He may be back after wife and kids one day. It happens far more often these days that marriages don't work.
He may just find that he would rather work out how to achieve his goals with you by his side. You may find that you could raise a kid or two together one day... all of you. Or maybe you will find that being his close friend and companion and occasional lover is a good place for you while he embarks on a marriage and kids.
All of this is possible if you all decide to embrace it all and not apologize for who you are and what relationship dynamic you create. The joy of age is that one seems to develop a back bone in these areas and there is no backing down from what one wants to do in life.. Perhaps this is why you think that it seems to be easier with age? Its not easier I don't think, just everything tends to be clearer as to how to achieve ones goals. The determination level goes up some how. This is my experience any way.
The main message I want to convey is that you can create what you want? Deciding what you want for yourself is the first step. Then ask your husband and this new man what they want... this is how negotiation starts. It takes time and patience. You have LOTS of time at your age and stage
. Poly relationships are like moulding clay... you can create what works for you, it just takes thought, creativity and keeping at it.