It's not babysitting when you are caring for your own children! I understand the resentment though if one parent is able to have "grown up time" and the other parent is always left with caring for the children.
Nightwalker, would you feel better if you were given one full day a week to pursue something of interest to you? To me that seems like a fair compromise. It sounds to me that you really need some time away to be yourself and to not be Daddy for a while.
This kind of thing still happens when people are monogamous. One partner can have a hobby or friend or job that takes them away from the family for long periods of time during the week. Is this other relationship healthy for your girlfriend? If it is try to shift your focus from it being another relationship to your need for time away as well and time as a family.
Everything will be ok in the end. If it's not ok it's not the end.