Well, I'm polyamorous and I also swing with my primary boyfriend.
I find that my absolute favorite situation is when we swing with a couple and then end up becoming good friends and develop a more intimate relationship with them, so we see them many more times. Sort of like two couples dating each other. I'm not sure if that situation would work for you, but it seems you may be able to enjoy the closer intimacy while your husband can rationalize it as a friendship. Best of luck!
Me: Mid-Twenties. Bi-sexual, but not bi-emotional. My open poly boyfriend: Wolfwood. My mono-boyfriend: Russo.
My blog; sexually-explicit posts locked under friends-only. Friend to read: Polygoodgirl