I feel great compassion for you. It sounds like there is a lot of suffering in this thing.
I will be interested to hear what comes next. In addition to some excellent advice from some of the more experienced members here, I would add that perhaps you and your husband are more interested in non-monogamy and swinging than in polyamory? I know people have established sexually open marriages where one of the ground rules is "no emotional involvement." I have no idea how this works, as I can become emotionally involved with my mail carrier when she walks by, but maybe this is a lifestyle that would work for you.
Then there is that pesky lying/cheating/spying stuff going on. I call this "the complicated dance." I truly believe people are in the midst of the complicated dance because they want to be. They enjoy the drama of it. They like the complexity. They appreciate the way it distracts them from taking a good hard honest look at their own stuff.
It doesn't sound like either you or your husband are really interested in emotional involvement with others. I think you both tend to use others to get what you want. This can be a recipe for real harm unless you find a way to be aboveboard and take responsibility.
And I don't know him and have never met him in real life, but the way you portray your husband, he sounds like a totally manipulative, abusive asshat with shitty boundaries and very little respect for you.