Op IMO there would be nothing wrong with saying my house needs to be MY safe place and that he can see her outside the house.
It would have been very unfair to invade my husbands space with my boyfriend every single night of the week. Heck my boyfriend had never been inside my home until Christmas day, and that was after we had been dating 8 months. The house is my husbands safe zone.
There is nothing wrong with needing your own personal time with out her there. You all need a schedule. My husband and I have one. I am allowed to see my boyfriend every other weekend when he is off and one of his days off during the week. The kids needs/activities come first. But my boyfriend is welcome to attend since he is included in the kids' lives too. For example my 6 yo wanted to go to the National Aquarium for his birthday, and he wanted both my husband and my boyfriend there. So that is what happened. We had fun.