I'm sorry that this has been your introduction to non-monogamy. I hesitate to call it polyamory given the lack of openness. One of the simplest definitions of poly is having multiple loves who all operate with honesty and openness in regards to the presence of each other. Given that your wife has had affair(s?), I can see how trying to shift to a poly dynamic would possibly cause you pain and resentment. Poly should not be an excuse for people to act irresponsibly and hurtfully towards those they love. It sounds like you've thought pretty honestly about what possibilities there are. It doesn't seem like an easy decision. But don't feel like you're the bad guy if you don't want to try and create a poly dynamic from this situation. There's a thread on how introducing poly to a relationship that's having trouble can look. You may find some useful perspectives on that when you're considering your choices.