There are just so many ways things can go awry! For instance, a married man may be much more interested in "polyamory" as a sex-style, vs a love-style, and the woman he gets involved with may want his love and not really be getting that at all, or very much of it. And this might cause her to express a desire for more of his time, focus..., while what she really wants is more of his love.
And, too, we've all been brought up in a culture where exclusivity has been held up as a measure of affection and commitment, with non-exclusivity held up as a "symptom" of love-lack and non-commitment / insufficient affection.
It must be obvious that these and a million other things can go awry, and that -- in our culture/s -- polyamory is a couragious adventure mainly for very good communicators and exceptionally loving (and emotionally/spiritually mature) people. It isn't for the faint of heart.