I would never equate poly with cheating, and I can see that is a sore spot with some - because they certainly don't want to be thought of as adulterers merely because they love multiple people.
If we take the cheating out of it (which, I know we can't) then it's just me and him looking at being open to loving more than one person. We're not there yet. First, because obviously, we need to get divorced or he needs to make sure wife is okay with it. Second, because we haven't really gotten to the emotional aspects of having multiple relationships. We both agree that if a person continues to have sex with the same person, even if it's not often, some feelings do develop, which are necessary for any ongoing "relationship."
We don't really want to be swingers - NSA, anyway. For us, it's more a matter of finding others who we're both okay with, and who are okay with us. I'm more picky about approving of partners for him, than he is for me. Naturally. So far, nothings happened on his side yet, anyway. He really hasn't had time. Right now, it's the thought that counts, I guess.