I am so sorry...
If I were you, I'd hug Perm, told him I loved him but I couldn't do it anymore, and leave. Maybe move in with Cue and L, at least until I could get my own place, but I'd be out and done. I'd get a lawyer and serve the divorce papers, make sure to watch my own ass in terms of the proceedings. (The court isn't likely to know about poly and open marriages.)
If he can't communicate to you about his needs, and refuses to grow, the relationship is stalled. He's the only one who can restart the engine, and if he won't then there's nothing else to be done.
I was in a relationship that feels to me like yours seems to be (though we didn't marry) for four and a half years, and it was technically open for the last two and a half of it. I loved him, and still do three years later, but he's mono and I'm not, I'd compromise and he wouldn't, and it was the most unhealthy situation I'd ever been in. He wouldn't let me tell mutual friends about A (my other boyfriend), imposed serious restrictions on A's and my physical intimacy, insisted on a DADT policy that wasn't tenable (I couldn't tell him if I was going out with A, and then he was constantly was checking up on me), and was hysterical every time any poly-related topic was brought up. The only times he'd even talk about being willing to compromise where when I'd tell him I was leaving-- and those offers were alongside vaguely suicidal threats.
I wish I could give you a more balanced perspective on this, but the best I can say is that when I was in similar shoes I threw them off and ran.
Thanks for being brave enough to share your story. Keep posting-- we're here for you.
"I was thorough when I looked for you, and I feel justified lying in your arms." - Chasing Amy