In a nutshell, he's not being honest. Hubby is not respecting the rules/boundaries you two have and is thinking with the wrong head. It also sounds like a case of NRE (new relationship energy, if you're not familiar with the term).
You sound very committed to your relationship. The two of you need to sit down and figure out the WHY behind his transgressions. Though he's going about it in completely the wrong manner, my first instinct is that he has fallen for her. For better or worse.