I have to agree with Annabel's original question... That seems like a very important question that goes hand in hand with the problems you're facing.
The problems (as I see them):
1. Oh lies, fails to use safe sex practices, and would not hesitate to cheat on you if you asked him to step away from the gf for a while. He doesn't really seem to be willing to rein in his NRE at all, fails to respect the bf's need for space, and fails to respect you through all of these things.
2. The bf feels rushed and would really prefer to keep things non-sexual until everyone (especially him) has their feelings under control and understood.
3. The gf also allows herself to be pressured into having sex with oh even when she knows it goes directly against the wishes of you and the bf.
4. You are feeling neglected by oh, feeling unattractive at his desire for gf while seemingly having none for you, and are overall frustrated with the situation.
To me, it seems like NO ONE is happy, except for maybe oh who seems to be getting almost everything he wants (you bringing up his errors is apparently bothering him, but he still gets to screw who he wants, when he wants).
I really don't see long-term potential here unless oh straightens up and respects people, gf starts actually doing what she says she's going to do, and all of you get on the same page about what is and isn't okay.
Even in a swinging relationship, respect is imperative. I've seen many swinger couples end their marriages/relationships because one or the other decided that they were above the rules of safer sex and honesty.