I think you're right Derby. I'm not sure if you've followed the whole thread, but that's the realization I had somewhere in the middle there.
Basically the answer is "no, I don't want to risk our friendship over it." Easy answer. It's honestly just nice that I can be upfront with my wife about having a close friend that I'm attracted to. I don't know that it even needs to go any further.