This sounds like an incredibly painful situation.
I'm a newbie but to me it sounds like perhaps he's not so much into caring relationships as he is into fooling around. I hope I'm not being unfair. I don't think you are reacting mostly to polyamory, I think you are reacting to your husband's impulsive and covert sexual behavior. It appears that way to me, I guess.
My intuition seems to say that you two need to find help with issues of marital trust and supportive relationship-building. I would recommend getting some counseling. I tend to think that without some perspective and self-monitoring he may manipulate any rules and bend any boundaries to suit his sexual whims.
I may be very wrong. Unfortunately I have only feeble male intuition to go on...
So I guess you have to carefully evaluate what I've written -- and what other people write to you -- because you are the one who truly knows what the situation really is.
But I do NOT think you are in any way wrong to feel hurt. Your feelings are valid. The first person you need to care about is yourself. That's always true in relationships, I suppose, but it's something to remember especially in your situation.
Good luck, and write more when and if you want.