This is your issue. Please don't try to make it theirs in any way, shape, or form. Seriously, relationships work best when personal issues don't get imposed on others willy-nilly.
Have you tried to unpack your feelings on the matter? Most folks experience a bunch of emotions at the same time, all knotted up into a little ball, that then gets regarded as a single bad feeling. In reality, it's a bunch of feelings.
The thing is, where that knot of feelings is difficult to handle, each of the individual feelings involved is a good deal easier to handle. If you can pick apart all of the separate feelings, you can deal with each in turn and free yourself up.
I suspect there are additional feeling lurking there because you've already got a good response to the idea of societal norms and "I should be good enough" stuff. What else is tangled up with those?
When speaking of various forms of non-monogamy...it ain't poly if you're just fucking around.
While polyamory, open relationships, and swinging are all distinctly different approaches to non-monogamy, they are not mutually exlusive. Folks can, and some do, engage in more than one of them at a time--and it's all good.