welcome to the forum. I don't see any problems with your life really, from what you say everybody seems happy including all the children.
I am also new to all this but I have discovered from being on here that nobody's poly is quite like anybody elses.
If it is working for you all that's great. Maybe your friend not being able to handle the sexual side of things with you will change back again later. You didn't say whether you missed that or not.