Thanks everyone for the advice. She and I and our spouses have been talking a lot about this.
My husband's opened up a bit. He says he does not feel he would ever want anyone other than me because, in his words, he'd "always be comparing them to [me] and no one else would compare" aww! But he also says it doesn't bother him if I date her because he feels he's always been sharing me with her (as a friend) before and as far as he is concerned this isn't too different. He also says he knows I love him enough that he doesn't have to feel threatened by her. He has never been a jealous or possessive type anyway so I can't say I am surprised but I am flattered that he trusts me so much.
Her husband is relieved at the idea because he feels he has more breathing room in their relationship. He works a lot so she's been a bit clingy with his free time in the past, because she didn't get enough attention during the day, and he feels like I would be good for her to help her get the attention she needs.
I think I am getting over my feeling of reluctance and being uprooted. I think I do want to give this an honest try. I will try and read more on the subject, and the advice everyone is giving helps a lot so I will continue watching this in case there is anymore. Gonna get some Kindle books soon as I have money again. Any recommendations?