Sorry to be so harsh, but having been in the shoes of the cheated-upon, I have very little sympathy for you. From M's husband perspective, you're the "other man". You've cooked up a lot of rationalizations about how you're fine with that, but none of that is going to fly with him. He wants monogamy in his marriage, and you and his wife are spoiling that. What if he finds out you're still communicating? Suppose he sends you a nasty email or letter and tells you to back off, or worse, confronts you in person? When my W had an affair, I made her cut-off communications, and then I contacted OM's fiancee and told her all the sordid details. There's nothing like the harsh light of reality to end the fog of an affair.
Suppose as a result of your behavior they divorced, and sent the lives of the kids into turmoil? Study after study has shown the incredibly traumatic upheaval that results in the minds of children who are suddenly faced with losing a parent, or being a pawn in escallating custody battles, and often faced with reduced standards of living.
End this affair until (and if) they work out their issues and actually make a decision to pursue polyamory.