I'm in the same situation as you. The man I'm seeing is someone with whom I knew years ago there would be an attraction between us if we were both available. Now that I'm divorced, he views us as both available, and I've really been enjoying his company, but I think deep down, I don't entirely regard him as 'available,' no matter how much he and his wife both do. Besides, which, he is not available for a relationship that has the normal course of things as possibilities. So for me, it has impacted my feelings. I very much enjoy our time together, I like him, but I simply don't want to spend my life going to bed alone while someone I love is going to bed with his wife, and the attraction that I believe would have become deeper in another situation is not going where I once thought it would, for me, if we were ever seeing each other.
Of course, I'm only 4 months into this, and I have no idea at this point what a month from now holds let alone, 2 or 3 years. There are some mono/poly couples who do very well.
He does encourage me to meet other men, and I don't feel badly about it at all, because he's right to do so.
Look up newtoday and dragonsbyte, who are also monos dating polys. As is (forgive me for very possibly getting the last letters wrong...) MonoVPHG who posts here frequently.
Please feel free to send me a PM if you'd like to talk.