I can see both sides of you and your husbands thinking. But in this case, my husband is not really the kind who is always flirting and being suggestive, so when it came up, I was a little shook up by it.
I'm feeling right now, that I am fine with them going to dinner Friday, but before anything goes further, I need to talk with him and maybe her. We started our swinging "lifestyle" slowly and it's been great, so I think that approach is what needs to happen here too. Some people can just jump right in the pool, other people have to dip their toes in and work their way in.