I understand where you are coming from!!
I am struggling with a similar thing right now.
Hugs, I think the tough part (well for me, anyway) is being direct with my partner on my views and how they could affect us and then dealing with a love interest at the same time.
Also difficult is while I think I can imagine what it would be like if my partner wanted to take a lover (I don't always see that word used here, but that's what I think of it as) the fact of the matter is my husband probably would not. I am his 'everything.' Which is sweet and also slightly infuriating as I feel like he should probably take charge of his own happiness instead of thinking that I do that for him.
Like you, I've always probably been of a poly bent, although I never really had a name for it.