I hope you protected yourself when you had sex with him... that is my first concern. He is not a man to be trusted by the sounds of it. Self esteem issues or not, feeling sorry for him or not, connecting again or not, he fucked up and is putting your health and the health of your family on the line. Well.... actually you would be if you are having unprotected sex with him.
I wouldn't be so sure your son didn't notice he didn't call. Kids remember and take in EVERYTHING, regardless of their age. He is, in essence, teaching him that Daddy is not good to his word. You know that already, but is it fair that your son know that? Is it not the goal to make sure that he trusts and can rely on the adults in his life. Every little thing counts in poly and with raising kids.... it's called integrity, your man doesn't have this critical pillar in his foundation.
To me the foundations of poly (which I seem to be going on about lots these days
) are integrity, respect, honest communication and empathy.... he doesn't have any of these and to me this means he is not practicing good poly ethics and is a cheater... an open cheater.
What you do with that is up to you, it could be NRE as GS has said, but that doesn't mean that you need to sit back and take it. You have some choices here, but really, if this were me I'd be done with him until it he grovels at my feet for a bit and makes some changes to his foundation. Trusting that would take a very long time for me.
It sounds like you have put in some hard work with him and he is throwing it out the window for a quick fuck with someone he barely knows. I don't care how much of a low self esteem he has really, he has been in therapy long enough to know that he has some issues and needs to be very careful and he doesn't seem to care.
Sorry, it sounds harsh, but I think at this point I would give him a really hard lesson and tell him he has to leave until he is willing to address these very major issues in your relationship and in your family as a whole. It's okay for him to see potential in loving another. What is not okay is how he is going about it over and over again. What is really a red flag for me is that you are letting him.