Okay, so I'm new to all this poly stuff but my husband and I have been swingers for some years now and are use to sharing each other with other people.
I am on the flipside of your story, however. I have found a sexual partner that has turned into something more to me. I love him, but I still very much love my husband as well.
Relationships evolve and sometimes old rules don't apply anymore. You yourself have said that you've even broken your own boundaries once upon a time. And from what's going on, it is clear that these rules are not working. Your husband is in love with another woman but he still loves you. I think the withdrawal part has something to do with your responses to this and because of these rules. I also think that is what's going on with the arguments. Maybe it's time to sit down and really focus on the rules and what your needs are from him. Maybe you both can come to an agreement, but I think you have to learn to accept this new love. It's not like he can just turn off these emotions he's feeling. . .