Thanks for all the insightful comments
This is my first time posting and I wanted to say thank you so much to everyone for taking the time to make such thoughtful responses. This gives me a lot to think about.
Something I noticed: I am getting the sense that some view poly as you just do your thing and I do mine. In my house, we are in a life-long relationship. We do not operate our lives quite that independently, or have one foot out the door. We discuss everything. We check in with each other before and after dates and make sure what we are doing is in keeping with comfort and commitment levels. We have been together more than 20 years, are raising a child, and are committed to growing old(er) together. So, we don't just say, choose your secondary or choose me. That's just not our scenario. Yes, he knows how I feel about the ex. Ad nausea. That is why he isn't pushing so much as just reminding me periodically that he is still interested in her and would appreciate me not closing the door to that. I seriously consider it every time it is discussed (a few times a year?) and say I am trying to get my head around it and here is where I have trouble, how can we get past that? No solution has become apparent yet. We talk and I say I will think about it. But whenever it comes up, I have such a negative visceral reaction, it doesn't go much further.
I do not think that my telling him how I feel is not making him resent me. He realizes I have legitimate concerns and that she (or the relationship I guess, to be fair) caused major drama before. He justs wants this because she is a fun fuck and low maintenance (which I understand -- very appealing). For him. He is not desparately in love with her and my saying it pains me for you to do this won't end his world. I would also add that if the roles were reversed, I cannot see myself refusing to end a relationship with someone who hurt my spouse so much and who obviously causes such pain in our relationship. My husband is first, always. And I am first with him. That's just how we operate.
I should also say that seeing people outside our marriage is more for dating enjoyment and less a "lifestyle" mandate. While we have caring feelings for those we date and love some of them in nurturing, non-posssessive kind of way, I would not say we are "in love" with them in any way close to the way we are in love with and committed to each other. So, our polyamory may differ from yours. I realize that some folks emphasize the "amory" more than others. Maybe you could describe me as more polysexual? I'm not sure the term. While I like having a primary secondary I really connect with -- that sounds awkward -- I may also have periodic dates with others, as might my husband (which I fully support), for whatever reason. New experiences? New connections? Doing things our regular secondary or primary is not into? Just plain fun?