I think you guys need to go your separate ways. Your new interest may be poly and that may be the way you decide to go with your life, but it doesn't seem like your hubby really is wired that way. Again, that's ok but it seems to me that the best course of action is that you and he go your separate ways and you live your life. JMO of course and good luck.
Live life to the fullest 'cause you never know if you're gonna wake up tomorrow!