It looks like he's staying for the kids, because he wants to continue to co-parent with this woman. Except for co-parenting and finances, he is not married to her in any real sense (or at least that's what he's telling you).
I say go for it. Don't get caught.
It's not poly. But that doesn't mean it's wrong.