Why are you still together? Srsly. It sounds to me like neither one of you are ready for a serious relationship, either mono or poly.
It may work best for you if you split and each attend to your own issues. Once you've got a handle on those, then you'll have a chance of working out your joint issues.
When speaking of various forms of non-monogamy...it ain't poly if you're just fucking around.
While polyamory, open relationships, and swinging are all distinctly different approaches to non-monogamy, they are not mutually exlusive. Folks can, and some do, engage in more than one of them at a time--and it's all good.