This happened to me ONCE......But I had justification IMO. Me and the woman started going at it...and I looked down, and realized she was on her cycle! So....yeah....I went limp and was a non-performer for the rest of the night while my wife was shagging away in the next room. LOL Otherwise, this has never happened to me..... BUT, it HAS happened to other men with my wife. Of course, this was in our "swinging" days. We would get together with a hot couple, and I would get together with the other woman and be going to town....but the other guy would get together with my wife, and it was "failure to launch".
This was COMMON on the first time we would get together with a couple. The second time and beyond, though, it was GAME ON and all systems GO for everyone involved.
I would say it is just first time jitters combined with your wife in the other room.