So, when does he ever think of you as a person and *ask* you if you'd like to try something? His hamhanded scheduling of events for you suggest he thinks of you as an accessory to his life, a toy or pet of some sort, instead of a person with the authority to make choices for yourself based on your thoughts and feelings. What an insensitive clod!
This may be a teachable moment for him. I don't see anything useful coming of it until you've had time to process and recover. I'll suggest simply telling him to back off and wait for you to contact him to discuss it. Once you're ready, then you can explain all the ways in which what he did was wrong.
When speaking of various forms of non-monogamy...it ain't poly if you're just fucking around.
While polyamory, open relationships, and swinging are all distinctly different approaches to non-monogamy, they are not mutually exlusive. Folks can, and some do, engage in more than one of them at a time--and it's all good.