This comes back to the one of the premises of polyamory. You can only be responsible for yourself. That night you took responsibility in as much as you used a condom. There was no way you could have know short of an inquisition that he was attached. And that would have really set the mood lol. He was responsible for himself and he has to live with those consequences.
P.S. how come I can never seem to find a woman who is just horny lol.