Thanks for all of the advice/insight! I have had a sort of emotional epiphany. Do I deeply care about her? Yes. Could I potentially be 'in love' with this woman now or in the future? Yes. Do I want to limit what she can do PHYSICALLY with other people? A resounding no...
I love just being with her. I'm not worried about her replacing me, well I'm sure I will always somewhere wonder if she'll find someone else, but I'm not worried about it. Either we love each other or we don't, and I am going to enjoy it for as long as I can. I'm going to talk to her tonight and tell her that I really am okay with whatever she wants to do while in swinging encounters with her husband. I want her to tell me about them afterwards, but just like I'm okay with my husband doing what makes him happy and brings him pleasure, I am comfortable with her doing the same. It is really the time commitment and the affection that she shows me that makes me feel special, not the sexual allegiance. As long as she continues to take the time out of her busy schedule to spend at least one evening a week with JUST ME, no husbands, then I am quite satisfied. I was just stressing way too much about taking the next step, when we've really been there for a while and hadn't acknowledged it. I'm going back to just going with the flow.