Mookitten, I don't see him as being very kind nor understanding towards you. He sounds like a big ol' dictator. People shouldn't be waving the poly flag while running roughshod all over the ones who care about them. Personally, I could not stay in a situation like yours, where I was treated with such disregard. Why do you even stay?
Yes, you are "allowed" to feel hurt. Your feelings are valid because they are your feelings, and don't need to be justified to anyone.
One important question: does he use condoms, and at least let you in on his safe sex practices (if any) while he is running around fucking all these people?