I act as I normally would. I want to be affectionate, I am. No worries.
I recall the first time I EXPLICITLY told someone that I was in an open relationship (I am poly, my wife at the time was open). He asked casually "How come L isn't here?" and I responded with a casual "Oh, she's got a date tonight."
The next line was "Oh, wow, I didn't know you guys were open!" Turns out, he was too.
Your relationships are your business. By engaging in them, people might question or ask. Asking is usually the fun part, and honestly, it's fun. Many questions are repeated and you come up with witty answers pretty quickly. Sometimes, people don't ask and that's how rumors start, but it's pretty easy to weed out the gossipy busybodies like that too, which is a benefit itself.
Connection is the path to passion. Passion is not the path to connection.