First,..you would need to Define the word "affair" as it is subjective and open to interpretation.
I don`t think it`s being used quite correctly here.
If affair in the simple logic is 'cheat'....then yes, I believe what you describe, is cheating.
Cheating, has nothing to do with what you do, how you do it, or who you do it with.
It has everything to do with being upfront and honest about any actions.
If you betray, sneak, hide, distort the truth from your partner, in a physical, evident way, that is a cheat in my eyes.
I believe the 'essence' of being poly, is openess, honesty, and communication. Lying to others, or yourself is not part of being Poly.
I hope this doesn`t come across as judgemetal. This is my own thought process. We each have to come to terms with things in our own ways.
The bigger thing, is what your partner thinks. If THEY feel betrayed, then THAT is what matters. If they don`t, then it is a non-issue.
I have been in a position, where I had a on-again, off-again romance, that my primary was not thrilled with. Some topics were very, very hard to approach, and I hurt when we didn`t agree. Still, he always knew what I was doing, and when. I never betrayed that trust.
If you simplify, and don`t get caught up in the details, it`s easy to remember what makes being poly different from cheating. Blur those lies, and it taints the wonderful aspect of poly.