Another thing this thread brought up for me is the importance patriarchy places on penetration.
For example, in many cultures [including prison culture] it is generally only considered "gay" to be on the receiving end of a penis. Also, the verbs for sex are typically gendered in a way that sex is always something that the person penetrating is doing to the person being penetrated. (Eg. Sam fucked sally; sally was fucked by sam. Ernie gave it hard to Burt; Burt took it up the a**. etc.)
So maybe that partially explains why I don't feel threatened by my partner having sex with a woman.