It sounds like you're about to have your first fight in your new relationship. Most (all?) relationships have fights. There's no reason to believe that, because this is a secondary relationship, the fight should look or sound different than a fight in a monogamous relationship.
What I would tell you as a friend is not to paint too broad a brush with your accusations. He screwed up Saturday. I doubt it was deliberate. If he concedes it was, you and he need to have a much deeper discussion. Based on what you posted, it was only this Saturday, not a pattern of Saturdays. Saturday was important to you. He needs to know that - perhaps in the context of a fight or meaningful conversation with appropriate emphasis.
I'm sorry you feel hurt, I'd probably feel the same way. He's not coming at this issue with your same history, so you might have to bring him up to speed about your background and sensitivities.