Cheating on your pregnant wife who's carrying twins - huge asshole or worst. Responsible for her happiness....No....Responsible for torment and heart break.... is that even in debate here. With all that this women has currently gone through to be hit with this now could easily be overwhelming. I'm guessing he is young or has a very different life experience. I'm sure the most polyist of the poly would have a hard time with the situation the op has presented. This is a very good question to ask my wife to get her input.
I agree with Mono and the others ..... protect yourself and trust your gut don't do this to please him because he by his own admission he wouldn't do the same for you. I would also make it clear to him that his vision of reality is distorted and that you have exactly the same rights as far outside relationships go and that you may indeed exercise that right.
Take of yourself and kids D