Hi and welcome, it sounds like you are going through a very painful experience.
You say these guys have been in polyamorous relationships before but maybe they have forgotten many of the differences between swinging and polyamory?
There is a world of difference and you need to make your boyfriend aware of your fears and take them seriously. Learn about polyamory yourself, educate your boyfriend and his girlfriend and set out your boundaries and needs, then negotiate.
That sort of sums it up in a nutshell but getting comfortable with polyamory is a process that can take a long time. There is plenty of help if you read the threads on here, also the poly/mono forum on yahoo and I started a blog (link below) to help me come to terms with my own poly partner.
All the best