I am a real hard ass when it comes to cheating.
Only because I have some experience with it and have been here a realllly long time
First of all, I'm really sorry this happened to you. Its a devastation that we have read about here over and over again. Have a look at the threads under the tag search "cheating." You will see you are not alone.
Cheating is not poly. Poly is consentual, based on integrity, open and honest communication and consideration (empathy). Cheating is completely the opposite of poly and about selfishness, greed, laziness, cowardice, deceit and can be addictive.
If he wants to live in a relationship dynamic that is poly then I suggest he start again. I suggest he tell the gf he won't be seeing her for a long time and concentrate on you and what you have in your relationship. He has a HUGE amount of integrity to rebuild. It could take years and he should expect it to be that long. He has no right at all, as far as I am concerned, to become impatient with you, disappointed its taking so long or push you to go any fast than you need. You have no right, if you decide to find a way to get to a place where you will be okay with his relationship with her or others, to stop working on your feelings of deception, threat of him lying again, feelings of betrayal and mistrust. He needs to prove to you that it will not happen again. Only you can say when you forgive him entirely and trust him again. Its you that will have to take the first step of trusting him again and he should know that that is usually a major deal.
He fucked up. You don't have to suck it up for him. He needs help. It sounds like he was deep into his deception if he was getting away with having had gf over for the occasions you mentioned. As Dinged said, "cheaters are only interested in satisfying their urges." I believe that also. It becomes an addiction of selfishness and greed. It takes away everything that you had and gives it to himself. All the wonderful stuff you two had together he chose to hoard and feed himself while he pushed you aside... it might feel like he gave it to her, but he actually did this for himself.
Again, I am sorry this happened to you. Now you know and you are in control. Good news is that it sounds like he wants to work this out and stay with you. Make the decisions that are going to be healthy for you in this new reality, for you and your kids. There are others that will be faithful out there if you find its just not going to work for you. There are good men that would love nothing more than to cherish and honour you with the values of integrity, honesty and compassion. Why settle if he is not willing to stop and see what he will lose. Hopefully he will open his eyes and see before its too late. It sounds like he has made a start... I am crossing my fingers for you.