Yeah, your partner explained how he wants his D/s relationships to be because he certainly isn't describing mine. I have free choice over who I sleep with, for one. My Daddy person does cater for most of my whims yet he is still my Dominant. We have a Daddy/babygirl dynamic which means I get to be a spoiled princess when I don't have to be a filthy slut.
I'm telling you this so you're aware that not all kink based poly relationships follow the harem pattern yours is and as you've seen, many people who get into harems do so from a place of weakness rather than strength.