My husband and I had a similar conversation and came to some good conclusions for us.
Initially it was:
Him: I don't want you with other men.
Me: I don't want you with other women.
I'm bi so it was no big deal for me but he's straight so that was an issue. So we talked about it and we learned some things.
For him: I am his wife and he's been cheating on by fiancees before and it all just hit a little too close to him. He's not proud of it but it's a gut reaction so he felt he needed to share.
For me: He has two children. A boy and a girl. When I have our first child it won't be his first. It'll be his first time being in the delivery room and going to all the appointments and being heavily involved (he wasn't aware of the first one till some time after the birth and distance was a factor with the second child) but it still won't be his first child boy or girl so I'm really anxious about him not being excited. Add that to the fact he's not a fan of condoms (he'll wear one if asked but generally that'll cut down on the sex for him...he'd rather go the "get tested together frequently and be faithful" route dicey i know...ick!) and I'm worried about someone else having his children, which is not something I'm willing to share. The mothers of his children have been really really mean...I'm 21 so in the beginning when I first met him I was a teen trying to handle a really adult situation ( two kids, two baby mamas, military) along with graduation and my dad being sick and they were CRUEL. And I'm just not in a place where I could share that again.
Basically (without the whole sob story) it boiled down to babies for us. It worked for us because 1) I'm not interested in other men anyway except for occassional instances and even that would be oral contact and 2) he wasn't really interested in other women; he's pretty solidly mono. Like...at the end of the day it wasn't the emotional contact we had issues with it was the physical and we got on the same pages about it.
Ask yourself WHY.