It surprises me that you decided to move out when you could have stayed with your wife and family, and try to maintain the relationship with the girlfriend. Especially if the wife was willing to give polyamory a shot.
One of the biggest fears of any polyamorous couple is that your SO will meet someone new, fall in love, and then end your relationship so they can live with the new lover. It's as if polyamory is merely a guise used to replace one lover with another.
How do you plan to convince your wife that any new relationship you explore is not going to jeopardize the one you already have?