You have stayed in a marriage that's not working far longer than I would have. One thing folks close to me learn quickly: Homey don't play them games. The "I agreed to this yet I want to back out" game doesn't run more than twice aimed at me--I don't tolerate liars, cheats, and other dishonest sorts.
When speaking of various forms of non-monogamy...it ain't poly if you're just fucking around.
While polyamory, open relationships, and swinging are all distinctly different approaches to non-monogamy, they are not mutually exlusive. Folks can, and some do, engage in more than one of them at a time--and it's all good.