Well, glad to know that your first threesome was so enjoyable an experience!
It sounds like your partner is more interesting in a swinging setup, rather than polyamory, though. One of the typical aspects of swinging is that there is an understanding/agreement that there will be no falling in love outside of the main relationship. With polyamory it's encouraged and desired.
It might be a very good idea for you to try to get some clarity about what is and is not expected, and what boundaries are there, so that you can decide whether this is for you or not.
"Listen, or your tongue will make you deaf." - Native American Proverb