it sounds like your relationship barely has a footing at this point. Swinging degrades connection in relationships as giving away our bodies to strangers for them to masturbate into and others to watch while someone does that, does nothing for self esteem and a feeling of togetherness with ones partner. I say this as I have been there. I didn't realize it at first, but swinging damaged me in ways I didn't realize until much later. If you are a person who can somehow waltz through having meaningless sex with others just to get off then all the power to you. I personally think no one really can. But that is just my opinion. I see it as our cultures way of continuing the cycle of using and abusing women and objectifying them. We seem to be more and more self indulgent in the sexual realm and swinging just personifies that. It's all about "me" rather than "us" or "others."
Poly is about love and respecting others, bringing people into the folds of a similar way of being, values and core beliefs. It has little to do with sex and everything to do with daily life, community, chosen family, integrity, honesty and complete openness and really having ones shit together.
It sounds like you and your partner have a lot to learn and I would suggest that both of you not see anyone until you have your relationship back on course. If a primary relationship is not strong, and I mean ROCK SOLID! then the whole structure will collapse and people will get hurt.
Your man seeing several women right away sends of huge red flags for me that he has lost connection with you and is actually just trying to find someone else. I would ask him to spend his time giving you attention and reconnecting rather than dating anyone right now. I would also spend some time healing from all your swinging and really checking in with yourself to make sure there has been no adverse affect on your sense of self worth. You need to really love yourself before going into poly. You need to love yourself in order to love others to your full potential.
I agree with the idea that you will need to do a whack load of research on this before even considering trying out poly. You've got a lot of work to do. Go do it.
Anyone want to be friends on Facebook?
Send me your name via PM