Well, there's nothing inherantly wrong in a poly person being in relationships with monogamous people... HOWEVER, that has to be done with the agreement and consent - in other words it shouldn't be something that he demands of you if you are not 100% willing to give it and be happy. To me that is a red flag that absolutely needs to be cleared up before you get involved with him.
And yes, the other major red flag here is the fact that you are doing all this without your husband's knowledge. It is assumed that your husband has the expectation that you two are monogamous (and I assumed that you made the corresponding wedding vows and promises) - so that would be cheating which is (by my definition) absolutely not what polyamory is.
"Listen, or your tongue will make you deaf." - Native American Proverb