I'm very sorry that you are in a difficult place right now.
Unfortunately, there's no other way to handle this situation other than with candid communication. It'd probably be a good idea to sit down and express your feelings in a way that doesn't seem like presenting an ultimatum or out of the blue. That's really all you can do, unless I'm missing something.
The best case scenario: He agrees to be monogamous with you or at least confines his polyamorous relations to solely sexual encounter.
The worst case scenario: He doesn't agree with you and continues to pursue relationships which involve more than strictly sexual attraction, but I'm confident that you already know that. This also makes me wonder about the possibility of him developing what you might consider to be an emotional affair because he'll still be sleeping with people who he's attracted to and he'll still be pursuing people who he's interested in and if there is an emotional attraction, a bond will develop. There are many more elements to sexual attraction than that and what could develop is an emotional relationship which remains untitled, all while they are banging. Would you want him to only have casual sex partners with whom he shares a continuous friendship or no connection at all? I know you said that he's slept with your best friend, but that is obviously a person you know and trust. (*I'm not saying it can't happen but it's unlikely to if your friend is loyal and so is he.) Regardless, remaining silent about it will not help.
Also, what kinds of interests are these? Recreational or sexual?
It's unfortunate that this is messing you up. I hope you feel better and everything works out.