It's normal jealousy struggle. It happens. Both ways -- whether the gender of the other lover is the same as you or different. Because in the end I don't believe it is about the other lover's gender.
I think it about your inner thoughts and beliefs.
The whole "If our hinge sweetie compares us, I'm not going to be able to compete" thing. Maybe you are ok with the female lovers because they have equipment you don't. They offer your lover a different sex experience and your thoughts run along the lines of "It's not even a competition there." But the male lover has "your" equipment. So you feel competition fears.
It works the other way too. Maybe you think you are a hot stud muffin and no other man can compete to your magical penis. But crap! Here come women lovers -- they have goods you don't! What if hinge sweetie runs off with one of them and you cannot even compete because you just don't have a vagina to compete with? Ahhhh!
See? Doesn't matter the gender. Whatever tape is playing in there -- it boils down to "I am afraid I cannot compete."
You state your worries/fears yourself -- things like what if is he's a better lover, has a bigger penis, etc.
What if he is blond, has three dogs and a dragon, he drives a vacuum cleaner to work? ( I joke to keep it light and try to make you smile, I'm not making light of your feelings or minimizing them.)
It just doesn't matter what he has or does or is. What is causing your discomfort is not the trigger (him being another lover). It is your thinking and what you tell yourself inside your head that is causing you your upset.
"I am less than, I will fall short..." kinds of thoughts.
To feel better?
Ask your partner to reassure you that your hinge sweetie loves you for YOU and wouldn't break up with you without warning or without giving you a chance to work on whatever needs are not being met.
And ask yourself to learn to cut it out and play a different
tape in your head.
"I AM good enough. Hinge loves me for ME and all I bring to the table."
Maybe these could help?
Try to BREATHE. Literally. Take deep breaths and relax your muscles if you feel stress/anxious/uptight. If you make yourself breathe and deliberately let go of any muscle tension, your muscles can give your brain feedback that there is actually no danger lurking. So there is no need to be chronically poised for "flight or fight response." Then maybe your brain can relax a bit in there.
In a way, it's good
that it's about the thoughts in your mind. Because you can always choose to change your mind!
You will be ok. You can do this work -- you can learn to overcome jealousy.
Hang in there!