I read your post and thought "how sad".
Although I'm 100%, completely, and unalterably heterosexual, it has been my experience after a lifetime of multi-partner relationships that some of the most rewarding, rich, and memorable experiences have been with my male poly companions. My best friends, the men I've chosen to be trustingly close enough to that I would share my lovers with! It is an awesome and brotherly thing, that your husband apparently wants to shut out. Very sad.