My advice? The other relationships of your wife's secondaries should not really be your concern. Let your wife figure out for herself how she feels about them. Getting involved will not have any positive outcomes.
Here's what you need to be focusing on: yourself. Work out, develop more of a social life, go to a guy's dating forum and brush up on your flirting skills. Get out there and meet some women. In the meantime, be happy for your wife's happiness and enjoy her company when you are together. At this point, keep your insecurities to yourself and be as supportive as possible. Channel any jealousy your dealing with into sexual energy. Don't act on negative emotions and "fake it until you make it" if you have to. If you can do these things, then you will get through this and your wife will appreciate it.
Be your wife's bacon. She's not going to eat bacon exclusively, but bacon goes good with everything.
Incidentally, you should check out sls.com to find GOOD swinger events. I never liked the clubs or any place where they allow single guys to come unattended. It's a free site and you'll be able to find all the parties in your area.
Just my 2 cents.