Reading between the lines
Are you considering a mono lifestyle because of this woman your husband has been seeing? It sounds as if he has the potential of a mono relationship elsewhere and that is why you are considering the change.
A poly trying to be mono is a very difficult thing. Here is a possible (probable) outcome of that. His potential goes away, you cut contact with your secondaries. You make each other miserable trying to be what you're not and either split or you go back to being poly with him a mono husband.
You say you adore each other. If you read some of the blogs of mono/poly relationships eg the Blog of the Mono Wife, Kat Tails you will get some idea of how hard it is for these relationships to work. I hate saying if I were you but I'm going to anyway...If I were you I would stick to who you are while you both have alternate supports and options. Love him enough to let him go then if he comes back to you and is prepared to be the mono to your poly he will be doing it of his own free will because of his depth of love.