All I can say is that your husband needs to get it in his head that he doesn't own you just because he married you.
I never understand it whenever I read (and it happens often) that a couple gets into swinging because the husband is okay with his wife sharing her body with someone else, but will not tolerate her sharing her heart. To me, that indicates an enormous devaluing of women as whole people! It says, to me, that the man sees himself as owner of a woman and that he feels entitled to dictate that she let her body be used by whomever wants it, but yet she doesn't deserve all the love she can find in this world -- real, genuine love that comes her way. Claiming ownership or possession over someone is an obsession based on insecurity, and isn't very loving at all.
Independent solo polyamorist seeking lover-friends willing to invest in friendship, companionship, and love, but without a need for partnership.
Never confuse commitment with exclusivity, love with ownership, nor sex with intimacy!
For me, it is far better to grasp the Universe as it really is than to persist in delusion, however satisfying and reassuring.
~ Carl Sagan