My bf struggles with anxiety issues. He has overcome most of them, and we have gotten to the root of what causes it through many many conversations and alot of ups and downs. But overall, it makes him feel as you say...through his growth and change he has said that he sometimes wants to go away and leave us both, start anew. He says he knows he really doesn't want to do that, but sometimes his anxiety makes him feel like that. We are giving him the space he needs to find himself more and more. and he is seeking help for the anxiety.
There is nothing wrong about finding yourself, finding new passions and opinions in life, and walking on a good and new path. You have found indepedence and freedom from some internal chains. Thats fantatic.
Question...If you are in a Poly situation, with the love and support you get from your wife and GF, couldn't you do some exploring and independent living/enjoyment within the world that you have created? Isn't that one of the great things about Poly?
And, you have a wife and 2 children. If you were fighting and living in an abusive situation, I would say leave now! but it doesn't seem that way. I just think you have more digging internally to do to find out about yourself more. Maybe find out what makes you feel like this about these supportive people in your life. You may be harboring some resentment towards one and its not disclosed yet. You may be missing that "drama" that you used to experience in your old, unhealthy relationship with mom.
So why not keep figuring yourself out and be more independent while you are in a supportive structure/environment?
I have to agree with BigGuy. If you take that walk solo, and drop the wife, gf, and loving family environment, they in turn may not be there when you return. Maybe in the next few months try taking a break from the two (can you get away) take a mini hiatus, and see how you are? Maybe you just need a little time to breathe independently and assess the entire situation from a clear, new head. If they love and support you, i don't see why they wouldnt give you that time.
BTW, I have found being Poly that soooo much communication exists, we have pow wows, and we hash out alot of things. I find that when talking, we dig up old, unresolved stuff, and we put a little attention and love/caring on it. It seems in by doing that, we have helped to heal that sore spot in their past, just by talking about it. After that, it never seems to be an issue anymore. Its amazing. Its like sped up group therapy in your own home!