To clarify, was this your lover's first poly relationship? You said it was the first for "both of you", but I'm unsure if that means you and lover, or you and wife.
If it was lover's first relationship, I'm betting she exploded out of the starting gate a titch.
I'd say, tell her you made a hasty decision, express interest in rebuilding, and then let her come around. You'll give her space, and have time to start working on your own insecurities in the meantime.
I also wonder if your wife is upset because she is the one who's been helping you deal with the fallout from this woman? No matter which way this goes, be sure to thank your wife and be cognizant of how much of your quality time is being taken up by the other relationship. I think if you can approach the relationship as a stronger, more confident person, this will put your wife more at ease.