Ah. I suppose if it started out with just both of you able to get other girls, then it makes sense if he is slow to warm up to the idea of you with another man, but it doesn't really make sense if he isn't willing to consider it at all and bars it forever.
I think that you probably picked a good time to start talking about it, when you don't have another male on the wings waiting to see if it is okay or not that is putting pressure on you to put pressure on your husband. If it is something you feel that you might want as an egalitarian part of your relationship, it is a good idea to start hashing it out before it becomes immanent and while you can give him a little bit of time to mull it over after letting him know it is really something you want.
Ultimately only you can decide what you are comfortable with, but if you are uncomfortable with a double standard, be prepared to have some headbutting and maybe even some counseling in your future if he has never had to challenge or work on his own jealousy issues regarding other males. It is something you want to probably approach delicately, but firmly and let him know it is something you expect him to seriously consider rather than just dismissing it out of hand or jumping to divorce as the first option.