Is the problem lack of sex or lack of poly females to have relationships with? Those are two completely different issues... Maybe the problem is that you have another lover but he doesn't?
It sounds to me as though this gentleman is seeking "happiness" elsewhere while taking you for granted... If what you say is true then I would say that he isn't communicating well enough to let you understand what the problem is.
If you were to carry out a trial separation maybe he would be knocked back to his senses and remind himself that he has a loving wife. I don't know what advice is best for you, but my gut feeling is that the harsh criticism regarding how obnoxious he has become would be better served by his friends than by you - that way the element of "you have a poly lover and I don't" will not have to be focused on.
Good luck : (
Last edited by Whisper; 08-02-2013 at 10:40 PM.