Was it a spontaneous event? Because I think that being separate, you will miss out on some of those things.
It kills the spontaneous nature of the moment to say, "Hang on, let's wait for Selene and Andulvar!" And then hang around for five minutes.
Unless it has been otherwise agreed to, should the others not get their time together in whatever arrangements they wish? Do you want to have sex with everyone all the time? Does that mean you love them less? I would try not to take it personally.
Sorry to sound harsh. I'm having a hard time understanding what gives you the right to insist on being included in everything. If you lived a similar distance from Andulvar, would you be angry at him for masturbating in the shower?
As RP said though, you will get more space. And I strongly suspect the benefits of this will be seen once the NRE wears off.