I am sorry about this. It doesn't really take anger to end the relationship, you can do it amicably, and be kind to both him and yourself. Ugly discussions wont do any of you any good, it won't make him change his behavior OR your hurt, and it wont leave you feeling good about yourself.
When my ex-husband and I split up, we could have made it ugly, but chose not to, so we still benefit from each others friendship years later. Maybe you could write a practice letter about what you need to say, and either give it to him or say exactly that, and not engage in further discussions after you've said the relationship is ending. Maybe have the conversation at a park or somewhere private but not in your home, so you don't have the chance to have hours long arguments that circle around and around and dont accomplish anything useful.
Happiness will never come to those who fail to appreciate what they already have.