He was a jerk to you. Cheating is cheating and that means lying, betraying and not following the relationship's establishes rules.
Your relationship probably had exclusivity as a rule. He decided not to follow it, and didn't talk to you about it or warn you or anything. That's just wrong.
Coming out to a spouse or other long-term partner can be very hard. You're afraid they'll reject you and you would lose so much! But this in no way justifies cheating.
He should have told to you about it first, preferably when he found out himself or soon afterwards. You have every right to feel betrayed and he hasn't shown himself worthy of your trust.
It might still work out if things change, but you need to make it clear that the least he could do is to be honest with you. He can't keep on lying like that, as much as he might be afraid of your reaction when he tells you things, not saying is so much worse!
I had a hard time talking about things with my husband at first, I was scared it would hurt him. But I realised that one of the big problems is lying. If you're open about things, honest, you talk and everything, it's much easier for your spouse to understand and for things to feel a bit more natural after some time.
How could you be comfortable in this relationship when you don't know what they're hiding from you? You need reassurance, since you're in the tough spot of being a mono (I assume) who found out her husband is poly.
If he had told you two years ago, you would have had all of that time to get comfortable with it before he met his girlfriend (or was he cheating all that time?). It's not too late though, if he's willing to be honest and open. In a poly relationship, you have nothing to hide because you're doing nothing wrong.