He acted idiotically. If it were me, I would have said to him, "Is it really too much to ask that you be present and attentive when you are with me? Do you even appreciate the effort I made to give you a special night? You can't even bother to control where your mind goes?" And then I would have left.
It doesn't sound like he is handling having multiple relationships very well at all. He really bungled that one.