I've had this issue before when I've had more than one partner. Even though I enjoy sex, I don't "finish", and while it doesn't bother me it seems to bug them immensely.
It seems to be a really hard sell to persuade a woman that--although I don't come--I still enjoy sex with her. I found it helpful to make a comparison to her sexual experience. Most women I've been with don't always come during regular sex, but still enjoy the experience.
I don't think you need to "cut down" on the number of partners you have. So long as you're happy and still desire sex then however many you can handle is the right number for you.