"A hard man is good to find," as Mae West once said, Aphrodite rest her soul.
Sounds like you're in love with a man with ED. Condoms or not, it's disappointing. Since you only see him so seldom, in your shoes I'd welcome ANY kind of sex he is able to provide. Outercourse is great, and toys help.
Keep your options open and keep looking for another lover locally who has a nice hard cock. Perhaps your husband has one!
My primary insists I use condoms with anyone else, no matter how clean and shiny he is, and despite me being 56 and definitely not fertile. It just makes her feel special, to know we are fully exchanging fluids intervaginally with each other. (She's trans, she is pre-op.)
Love withers under constraint; its very essence is liberty. It is compatible neither with envy, jealousy or fear. It is there most pure, perfect and unlimited when its votaries live in confidence, equality and unreserve. -- Shelley
me: Mags, 59, living with:
miss pixi, 37