I think that you and your husband should talk more about what his fears are at you having another man, if that is what you are interested in. Maybe you can work together to put things in place so that those fears won't ever become real, but so you can still explore the relationships you want to, while he explores what he wants.
Please check out The Birdcage - an open, friendly Polyamory forum for all parts of New York State
"Listen, or your tongue will make you deaf." - Native American Proverb