I hate to say this, as I like to think kids are open to different dynamics but.. I'd just keep poly and other relationships out of the house when it's affecting the kids that much.
I am curious, how long have you and your girlfriend have been involved, sounds like about a year? Having somebody spend the night two months after letting your kids know you were dating sounds like too much too soon. Your kids already met your girlfriend? Did they like her when they thought she was just a friend?
I also would hope you aren't introducing partners too soon, single parents also often make sure they've a "serious" relationship before introducing a SO to their kids. And well..yes I think it IS selfish to feel the need to parade your girlfriend around under the situation you describe. Go to her house for overnight dates, let your wife show your kids she's fine with it.
It's not necessary to acknowledge her as a romantic partner when she's around the kids, or to be physically affectionate. You've told them, they know, to them it's just as if you're grabbing your wife's chest in front of them, they don't want to know, they don't care to know, they really want you to just shut up about your love life because it's repellent because you're their parents. Doesn't mean you have to stop being poly, but I sure remember hating being aware of my parent's cheating & swinging. Was NOT my business or something I needed to know. I was glad to know people were my parent's friends - what happened behind closed doors was not my business.
Happiness will never come to those who fail to appreciate what they already have.