Been there. Done that. Got the t-shirt. If there's any warning I would share, it would be this: although you may be developing poly and compersion-like feelings, if you haven't discussed it, I mean the four of you together, then you cannot assume you have a developing poly relationship.
The annoying encounter you described sounds like 100% normal swinger behavior... Be careful about the emotional risks you take while you lack information about how the other couple views things.