Well, all of this, as I see it (though of course I can only tell from what you're sharing here), has nothing to do with being poly, but with seeking for an excuse to do whatever he likes. Yes, you are right: this is a major crisis, but the problem is not polyamory (since I can't see something like that), but trust and honesty. In any relationship it is not enough not to "technically break any rules", because it should not be a formal act like when you try to bend a law into such a shape that you will benefit from it - everyone knows that this law wasn't designed with this specific intention, but the wording allows you to apply it in that specific way. But though no one can say anything against it, it's still cheating.
I am sorry, but this is everything I can tell you: the way you describe it, it sounds like a really unhealthy relationship. And it has nothing to do with polyamory - maybe in a weird formal way, but ask yourself if it makes sense to formally love someone...
"Life is what happens to you while you're busy making other plans." - J. W. Lennon