Wow. I have been in the same place you are (emotionally) just very recently, cruftnot. Everyone is different so please note that I only speak from my own experience here. I noticed a few things...
and was asking N for permission to embark on this new relationship.
"Permission". Is she the Queen of the relationship? Considering her feelings is important, yes, but putting it in terms of her allowing you to relate to someone else is only going to make you resentful of her if she doesn't grant that permission. No one is forcing you to stay in a relationship with her. No one is forcing you to leave her. It's your decision what you do.
This hit home for me when I realised that I needed to start looking at what was best for "us" for my relationship with me and my partner. I want to be with him - I want to be with him without a speck of a doubt - but the way I was thinking wouldn't allow me to be happy. I now see our present exclusivity (we've been together only 1.5 years and have been exclusive the whole time, mind you) as something that "we" need, not just something he needs, to build up our relationship, communication, trust and safe boundaries with relating to each other - let alone other people.
I'm not telling you to dump your secondaries, please understand, I'm saying that a change of perspective, a conscious decision to feel positively about the necessity of the situation as it is and accepting it rather than fighting tooth and nail to have my needs met (while also not sweeping them under the rug) has helped the relationship and my own mental health so so so much.
(she has PTSD and was working through some anger management issues)
She has Post Traumatic Stress Disorder and she's also working with you on opening up a relationship (however haltingly) while working on her mental health? Wow, that is one brave woman who obviously really wants you in her life. My partner has a severe Panic Disorder and a few other mental health problems, which he is working very hard to recover from, so I can empathise a little with your situation in that regard too.
It was very frustrating when my partner would say things like "Okay, you can be with him" and then I would ask if he were sure and really push to make certain that he could handle it but then he would crumble and tell me how guilty he felt for not being able to "make me happy" by being comfortable with me having another lover.
The reason that this happened, I now believe, is that I was dragging him along down a dark path that he wasn't ready to walk. Now, as I understand that being in an exclusive relationship doesn't magically transform me into a monogamous person (just like having a male partner doesn't make me straight), I feel a lot more comfortable just focusing on my relationship with him.
Wanna hear the weird part? The more comfortable I am with mono; the more comfortable he reports feeling about poly. No one is forcing anyone any more. It's really liberating.
My compass for when it's time to leave is: am I still willing to work on this? Are they still willing to work on it? - as long as the answer to both of those questions is still "Yes!" then I'm in. But I'm the first to admit that I like to take the hard road in life. Or, rather, that a challenge has never deterred me from love.
Sorry if any of that sounded critical, I know it's hard to do tone via black and white text, I truly do wish you and your family (secondaries included!) all the best.