Even though you're poly, your husband has a one penis policy and also can't even stand you commenting on a cute guy on the street?
For me, one of the big benefits of being poly is the basic ability for both my gf and me (who are both pansexual and gender variant, btw) to comment on cute people on the street without getting all bent out of shape.
Heck, that's a good thing for even mono couples to be able to do.
I'd say it's time to negotiate. Maybe get counseling, for yourself, or both of you, if he'd agree, to help navigate the waters. You can't really practice healthy poly with low self esteem. Also, most women don't like to practice poly if there's a OPP, since that is inherently sexist.
Your h is so insecure about practicing poly he wants you to pretend you're never even attracted to a man, and even has issues around you possibly being penetrated by the clit of a transwoman? (Many transwomen don't like to penetrate someone with their clit anyway...)
I'd say his hurt and his being bent out of shape is his problem. You do not have to kowtow to his attempts to control you. You could make your own decisions and let him deal with it as best he can. Maybe he will learn something. Maybe he won't. Maybe he'll continue to be freaked out by the inadequacy of his penis. Maybe he'll learn that his penis and skills are just fine. Maybe he'll improve his skills and start working out as well.
You make your choices, he makes his. You don't have to tamp yourself down to boost his ego. If you're really poly and want the freedom to choose your own partners no matter their gender, you'll be miserable until you are firm in needing that need to be met.