I felt like this a few years ago. I was already in a serious relationship with Peaches, but it was a mono relationship and I was just TERRIFIED of the poly feeling I had. I was afraid I'd lose him, hurt him and stay alone forever. Because it really DID seem to me that no one around me, in my circles would accept this kind of relationship ever.
Turns out one happy day Peaches mentioned poly to me and said he thinks it's how things should be.
Then I thought I would never find "the second partner". Turns out that when you're not really looking too hard, there's always this amazing someone who really thinks "hey. That could work".
So, poly people are there, somewhere. They are just mostly being REALLY low profile about this, because, like you, they are afraid of label and prejudice.
A good way to start the poly subject is to state that you don't believe people should be treated as objects or personal property. That you'd let your partner be free to experience stuff and that you think jaleousy is bad.
For me, making this about your partner always sounds better than saying right away: I like to date more than one person at a time. But have in mind that this is how to START. It gives you a chance to explain poly better without the person closing up to the idea right away.
Me, female in a V with Peaches, live in boyfriend and CC, boyfriend.