I normally don't follow this thread, but I was drawn in by your name Mookitten. When I told my husband, Runic Wolf, about your post, he said that what your boyfriend is doing isn't poly and it's pretty fucked up, iho.
Yes, you knew you were going to be dating a poly person, but while poly can include one night stands and threesomes, it generally doesn't include orgies. In poly, you don't get your own way 100% of the time; it takes work and communication and a willingness to listen to your SO when they are struggling and take their needs into account.
IMO, if he was serious about having a relationship with you, he'd want to return to the place where you met together, as a couple. Not tell you to go with your friends so that he can appear single and get fucked more.... sorry, but I'd kick his ass to the curb. You deserve better than that.