I think there's a certain amount of this you'll just have to get used to putting up with. I find that somewhere around 20-30% of the people I engage with have similar reactions of the sort you mention.
I take the route of being very upfront about poly in my online dating profiles. Ask people not to message me if they're not poly or interested in poly. And, I explain what poly means to me directly (helps clarify what I'm seeking).
I believe this directness reduces my prospects. But, also reduces my headaches from what used to be 50% cheaters and 20% converters (who think I'll change for them) down to that 20% or so that I still have to weed through.
Net. Net. Very useful.
Also, have the benefit of an active poly community out here. This helps get out and connect with like minded people. Although I don't go these events seeking partners, it helps remind me that I'm not insane. That would I seek makes perfect sense. And, keeps me grounded.
Male, Straight, Poly
“Instead of getting better and better at avoiding, learn to accept the present moment as if you had invited it. And work with it instead of against it. And making it your ally rather than your enemy.”