I see a pattern of your husband doing what he wants, and you reluctantly learning to accept it...
...or just plain putting up with it. What is going on with that?
That pattern is mixed thoroughly into your relationship. My guess is its been there your entire relationship. That will have to change to respect and balance before anything poly will work. The girlfriend probably sensed the "I can do anything I want" part of your husband and her innate bad attitude toward "the wife" decided it was a great opportunity to move in on you. That's why I say you both need to be in a place of respect for your marriage and balance. Both of you includes him, btw.
I agree you two are not ready yet.