Matt, it's completely ok to leave a relationship if it's not bringing you happiness.
If there is an aspect of your partner's personality that you simply can not accept, you are making the right decision.
Also, on her side... It's not nice or healthy to be in a relationship with someone who can not accept that very special part.
There's no point saying "I love every part of you except that big bit which is so very integral to who you are"
Icky for all. You both deserve unconditional love. If you can't give each other unconditional love and the cost of continuing to try is so painful.... You can stop.
That's a perfectly valid decision to make. Don't beat yourself up about it, if that's the decision you are making.. Be kind to yourself and accept that's the path you are choosing.
It's ok !
Having been through a break up that involves children, if you continue down the path you are choosing I hope you can pick up harm minimization sooner, rather than later.
If you can manage it... A few days of peace and self love would put you in a good position to manage the difficult times ahead.
Concentrate on basics if you can - sleep and a good diet.
Constant stress, lack of sleep etc... That stuff always reduces our capacity to make good decisions.
Get yourself first into a position where you are most able to make good decisions.
Wishing you all the best.