It doesn't matter that you all came up with this agreement together. It really doesn't. The reason most have continually harped on the agreement itself and not the fact that he's not respecting it is because something isn't working with the agreement. How many times does he have to "accidentally" breach the boundaries of the agreement before discussing it and restating the same rules stops being an option? Why are you and him the only ones in this discussion (apologize if this isn't the case, just going on context clues).
The definition of insanity is doing the same thing and expecting different results. You need a new agreement already.