Further thoughts... possibly the best way to phrase this would be to focus on what you need for your family rather than making it about the girlfriend being there too much.
In other words, you need (e.g.) a couple nights a week and one day on weekends that is designated family-only time. Then it's in no way about the girlfriend or your wife's polyamory, and in every way about your family's needs. Rather than being exclusive of the girlfriend, it's inclusive of your family. Point of view is everything.
“As I am sure any cat owner will be able to tell you,
someone else putting you in a box is entirely different
from getting into a box yourself.” —bisexualbaker