It is hard to know what to recommend without knowing what S is thinking. My guess is there is probably a communication issue or some deeper issues that have not fully surfaced.
Does he have issues with feeling like he is being controlled? Does he think he is spending enough time with you? How does he explan the lack of sex with you?
I think if you went to a poly friendly counselor, it would benefit the relationship. My instinct is that if M were out of the picture, you would still be having issues.
I wish you well.