How is he ADDING to your sense of security by telling you he's going to ignore you and your discomfort and go on to get his jollies anyway? It seems dismissive/ inconsiderate. No wonder you don't feel stable. He doesn't help create emotionally safe space for you.
See, I don't think that being firm about the fact you will not be permitting vetoes is a bad thing at all. The OP should take a leaf out of his book when it comes to establishing boundaries.