I had two thoughts while reading your OP:
Not feeling good enough did not start with your current relationship, so look out for ghosts, here.
The only way to get the victory over fear is to charge at it and walk right through it. This is usually painful, but will help you find the truth about what you want and don't want in your life. I am sure you don't want to live with fear, so address it with yourself and those you love, and trust yourself when the answers come, even if they are unexpected.
"Rocks will open and make a way for the lover."
~Hazrat Inayat Khan
I love Catfish and Charlie.