OP, the questions I'm asking myself about what you've written are these:
1) Is he having an instability or insecurity with his wife that he needs to work on first?
2) Are there trust issues between him and his wife regardless of whether there's another potential partner in the pic?
3) Is the "homewrecker" sensing the insecurities in the marriage (if they exist)?
4) Is the OP willing to start a dialogue about it, especially about why he wants to hold on to a friendship?
There are other questions but those are the highlights. Basically, they are questions I would probably ask myself if in your shoes.
I have been in your shoes... or, actually, in the "homewrecker" shoes (although that wasn't my intention at the time...) I'll leave my story for another time, but basically what happened is that I realized that it seemed to her like I was trying to homewreck, when actually that was the furthest from my thoughts. She was married with kids, and he was gone all the time with work, so there were insecurities there, and I sensed them, but not because I wanted to steal her. I wanted to be with her and I would have happily met and befriended him but he was very traditional and the issue was cut and dried. One day she just cut me off. But can I really blame her? No... Worst feeling in the world, the idea that I was hurting someone's relationship.
I hope you can get it sorted, whatever the issues are that are going on between you and your spouse, or you and this other girl.