You sound like you're doing pretty well with it. (Yes, it is ego, insecurity, and cultural conditioning--but everyone's got those things).
The simplest solution would be to become more familiar with other men who are comfortable with their wives having boyfriends/lovers. Join a poly social group, read & post more on poly forums, etc. Find yourself a community that includes men in your situation who can model the idea that they aren't less of a man.
Single, straight, female, solo, non-monogamous.