RP is a very wise woman and I'm coming in on this very late but I have managed to find your original post. I'm afraid I haven't got time to go back over everything else and I actually agree with RP on the children thing. As an aside Z's kids are 11 and 12 (girl and boy) and they don't live with us either. He would not expose them to a multitude of different partners even if he had them, which he doesn't. He does like them to know his SO as a good friend.
But the real issue here is you and your wife and I'm ashamed to say that you remind me very much of me and my ex-husband.
Ironically in my current relationship I'm the mono but when my marriage started falling apart it was me who wanted to be poly (although I didn't know much about it). I sound very much like your wife and you sound very much like my husband. He decided he didn't want poly for the exact reasons you've stated. He had all the responsibility and my "boyfriend" had all the fun. We split up over it.
But looking back, if I knew then what I know now I think I could have made it work. We had a conversation very much like the one you described in your initial post and I couldn't answer any better than your wife. That's why I let/made him go. I felt I was being selfish and unfair to try to hold onto the marriage under those circumstances.
But what I was going through was just NRE fuelled by a desperation to be loved for who I was and to feel exciting romantic love again. I had started raising my husband's kids when I was 19, I had two of my own in my twenties and then my husband's eldest daughter virtually deposited 4 grandchildren on my doorstep because she was so ill. By the time I was in my late forties I was totally burnt out. My husband had his own coping strategies and difficulties but we just couldn't get it together.
I hear all the resentment, hurt, anger and disbelief in your posts that my husband had and it's affecting me but I don't quite know what to tell you. My teenage daughter (two daughters) also hated me for probably a couple of years. She had lived through the worst of her parent's relationship and she blamed me because she saw me as the one who was upsetting the apple cart. We had counseling which helped but I had to "wear the hair shirt" as the counselor called it because as the "child" in the situation she could only deal with her feelings and not be expected to understand mine while she was in such a state.
From my perspective my ex made everything worse between us but in hindsight maybe she needed him to get through. It is very difficult for a mother to stand aside and acknowledge that in order to be true to herself she has to almost relinquish her relationship with her child.
I don't think we'll ever be close. I think she is her father's daughter and my first born is closer to me. We are much better now and I think we are all happy. But it was a couple of years of hell.