I think the times it's used as an accusation in poly contexts is when people act badly in order to preserve their privilege. It doesn't mean they're guilty of anything for having the privileges in the first place. But it doesn't justify acting badly.
For example, if two people were in an established couple and got used to the perk, chances are they know if they come out as poly or as part of a triad, V or other configuration, they will lose the perks they got from being "normal". They will be considered weirdos and so on. Therefore they might try to still appear as a couple, in effect hiding the relaitonship(s) with any new partner(s), since they need to "pick one" to introduce as their partner, and usually many people already know one person, and it's simpler to keep the same person for everything so you don't forget who knows who, or run into trouble if two people who know you as dating someone different run into each other and talk about you.
As a result, a third person is excluded from sharing privileges you get from being in a non-hidden relationship (which contains some advantages listed above, but also things like being able to hold hands or kiss in public, being able to talk about your partner(s), etc) and still gets treated as single (if they don't have an "official, out of the closet" partner too) even though they are not.
When their partners are invited to something, they're never the +1, because it would be weird. When they see their partners and other people are present, they have to make sure not to kiss, hold hands or use affectionate language. They might have to hide their relationship from their friends and family, too, if that's needed so the couple can stay in the closet.
All of this would happen because the couple wants to keep their privilege. They don't want to have to deal with discrimination. But they're fine with their partner having to deal with all the crap, so they can keep getting benefits they got used to. That's not okay.
Simply having the privileges is one thing, and it's neither good nor bad. In many cases, a privilege is really just a lack of being treated like crap, and it should be everyone's case rather than being restricted to some people.
Wanting the privileges is a different thing, and a very understandable one. Wanting to be treated with respect is pretty universal I would think.
But throwing someone else under the bus for the sake of keeping your privilege, and then denying that you're doing it, because you might not even realise it, that's when you're being shitty. Because you met someone second rather than first, they don't get to be introduced to your friends, family, coworker as your partner. They don't get to have dates with you in places where you could be seen. They don't get to talk about you to their family, friends and coworkers. They don't get to be invited to important events in your life. Had you met this person first, at that stage of your relationship, with the same level of commitment, they would have access to all those things. Instead, they don't.
Some people look at it, and instead of saying "this suck" and seeing what they can do to be more fair, they react by saying "Well you should be happy you get to be with me at all!" It's quite natural to want to preserve your privileges, especially when they are things everyone should be entitled to, but it's not fine to treat other people terribly because you're too afraid of losing them, and it's important for people to notice they're doing that, and that it's NOT "normal".
Treating someone you've been with for 10 years differently from someone you've just met is normal. Treating someone you've been for 2 years less well than you did your first partner after 2 days or 2 weeks or 2 months, simply because they didn't come first, and expecting them to be fine staying at a "not even as many rights as you'd have if I'd just met you and didn't have an other partner" for the rest of the relationship is not, in my opinion, fair or realistic.