I agree, it sounds like depression. It sounds like you truly love your wife and want to help. I think the first step to help your wife though is to get her to a doctor she needs to be checked out both physically (to make sure there isn't an underlying medical cause) and mentally.
As someone who suffered for a long time with not only excessive weight gain but severe depression and loss of all sexual interest I can honestly say playing the blame/guilt game isn't doing anyone any good at this point. At the same time, your wife needs to be willing to let you support her through this. It's not fair to shut you out. It's also unfair for you to "let" her continue as she is.
It took a year of extensive testing to find out what was wrong (turns out I was suffering from fibromyalgia). So don't expect this to be an easy road for either of you should you chose to take it. But after getting help I realized I wasn't alone and it was fixable. Both the diagnosis and support were huge eye openers for me. I made major lifestyle changes which helped me lose the weight, get healthy, and recharged my libido. (I've been symptom free for 6 years now)
As for the affair, it sounds like it's hurting everyone (you're feeling guilty, D probably knows something is up which is feeding the depression, and N by being the "other woman" is being put in an unfair position). I suggest calling it off for now if you plan on helping your wife. Now is not the time to try to push a poly life on either of them because they aren't open to it. Once everyone is back on the path to being healthy and secure then it's time to reevaluate what you need and, if it's a poly-life, start discussing things with them.