I have a few random, somewhat coherent thoughts on this... Maybe you can get something from one or more of them.
-It doesn't seem arbitrary to me. Why? Because threesomes/swinging is something that you do WITH YOUR PARTNER, so it could seem more like a hobby or a pastime than casual sex.
-You meeting men on your own and sleeping with them on the first date could potentially pose a safety issue. For yourself if a man has a side to him that you didn't expect, and for you AND all of your current and future sexual partners if the man isn't upfront about any potential STDs he may have. Herpes and HPV are two from which condoms can't properly protect you. The physical danger could easily be discounted in the swinging setting since you have Wolf with you.
-If you think it stems from insecurity, have you tried to figure out what he has to be insecure about or why he suddenly feels insecure?
-What were his thoughts when you first started the open relationship with him? Was it that way from the beginning or something that was negotiated after you two had been together for a while? Is it possible he's been uncomfortable with it for a while but has just now felt comfortable enough enough to voice his discomfort?