I have to agree with JaneQ alot.
It doesn't really sound like the two of you are 'using' each other any more then any relationship. It does sound like both of you have a healthy respectful relationship.
The relationship sounds pretty symbiotic to me.
Now I will also agree on the thought of you finding 'the one girl for you'. Honesty in any relationship is important, the more so in more complicated relationships. So to avoid possibly very hard feeling later on, all THREE of you need to be clear on what each of you expect out of this relationship.
If all three are expecting a causal 'friend with benefits' but not life/soulmate partners for life, thats fine.
If all three of you are expecting (or hoping) for life-partners, that's fine...
However, if one (or more) of you are looking for life-partners, but the others just are looking for casual break of routine, low-commitment... then folks are going to get hurt, unless they understand that.
You do seem to care about your friend, so it is more then just 'swinging' and casual sex. So my suggestion would be first talk to her, about your relationship and what both of you want out of it.
Could she be the 'one girl for you'?
Is she looking for another long term partner, or somebody to fill gaps that her husband doesn't but isn't looking for deep commitment.
Once the two of you understand what the two of YOU want.... then I would suggest talking to the husband (either as a threesome or possibly you and him alone).
Even if he isn't poly, if approached delicately and with it clear that HIS feeling are important, that by caring about HER, you care about him (at least in so far as he makes her happy also).
Especially if you lean towards the long-term deeper relationship, inquire about how you can help HIS relationship with her. Would you consider helping 'her' around the house, so you could spend time together later on... Now, how about. helping HER around the house, so she could spend time with HIM... Show that you want to help him and her have a good relationship, can go along way towards helping YOUR relationship with both of them.