I'm not really in the camp that poly relationships automatically involve agony. They definitely involve work but I've yet to know any worthwhile relationship that doesn't involve work. But it's the kind of work that I love doing.
So I guess it depends on where the agony is coming from. Quite frankly, if there's agony involved in any of my relationships, I'm pretty sure they're not good relationships for me.
If you're ok with the poly stuff, then it strikes me that there's no problem. But if you're viewing the poly as a source of agony in your relationship, then I would give that some serious thought.