I am sorry you h's libido has dried up to such an extent, and despite your desire for him, he won't seek help. Not even a little Viagra? Sheesh.
So, he'd given you the go ahead to have other sexual relationships. As a former swinger, you felt you could find sexual satisfaction elsewhere, while keeping your heart true to your husband.
However, now you feel you are in love with you sexual partner. Sex does deepen bonds. Many swingers do things to actively prevent this, such as never fucking the same person twice, never kissing, etc.
What can I say but be completely open with your husband? Tell him you're in love with your sexual partner. You are tired of DADT, you feel a need to tell him and let the chips fall where they may. Go for it! Courage!
Love withers under constraint; its very essence is liberty. It is compatible neither with envy, jealousy or fear. It is there most pure, perfect and unlimited when its votaries live in confidence, equality and unreserve. -- Shelley
Mags (poly, F, 61) loving miss pixi (poly, F, 39) since January 2009, living together since 2013
"Master," (mono, 36), miss pixi's Dom for 3 years