There was a letter to Dan Savage recently about a man who couldn't get it hard for his mistress.
Now, this is poly, not cheating, and we all know that here, but maybe the problems that guy was having ("I'm doing something wrong" > anxiety, stress > no hard-on) are the same you are having.
Even though you know your wife is fine with it, maybe you've been conditioned to think it's wrong and have a hard time (well, not so hard) getting past that.
Just something to consider.
Me: 31F, straight
Seamus: My husband, 32M, straight
Dragon: My boyfriend, 29M, pansexual, married to Fox
Fox: My boyfriend, 29M, homoflexible, married to Dragon