My husband and I started out this way...sort of. He told me a couple of times he would be ok if i slept with other girls but I didn't want to have a one sided poly relationship because I wasn't ready for him to sleep with someone else. It took about a year for me to wrap my head around this idea.
I would say both of you should maybe sit and talk this out, at length before either of you act on another relationship. That's what we did and by the time we ended up acting on our poly desires we were able to stay strong even through a lot of issues with girlfriends...