I'll be honest and say that I've definitely spoken out about what I see as red flags in certain types of poly, but I've never saw them as "not poly". And yeah, I've spoken out against the type of poly you seem to practice going by what you said, but my argument isn't anything to do with that arrangement not being poly more about why I think some people gravitate towards it. But that's off topic here.
Long story short, if you have or have the potential of consensually obtaining and maintaining more than one loving union at a time in your relationship style, and you want to be poly, you're poly. Nothing to do with how much sex you have, who you have that sex with and under what conditions. Just about the potential of multiple, simultaneous loving relationships.