Knee to nuts. End friendship. Do not tolerate his assholery ever again. Tell him you won't stay in touch with him or have anything to do with him because he assaulted you. Do not get into a convo if he starts to argue about it - state your case and walk away.
Unfortunately, group sex and having no boundaries are what many in the general public think polyamory is. I remember once having a conversation with a guy I had met in a bar. When I mentioned polyamory, he was all, "oh that's so cool" and then he started talking about threesomes and how much he digs bisexual women. I told him I am straight and not into group sex, that I want multiple relationships with men, and a look of horror and disgust came over him. I saw that, in his eyes, any woman who wants more than one penis in her life is nothing but a trashy whore.
I no longer use the word polyamory anymore. I talk about exclusivity/non-exclusivity, developing loving relationships, and commitment to honesty, but I stay away from the word polyamory just because of the misconceptions out there.
I'm sorry this happened to you. At least now you are prepared for the next dirtbag who tries something like that. Remember, knee to nuts.
The world opens up... when you do.
Oh, oh, can't you see? Love is the drug for me. ~Bryan Ferry
"Love is that condition in which another person's happiness is essential to your own." ~Robert Heinlein