Sigh, I feel like I am reading more and more threads here and elsewhere where people (usually men) are using poly to be the new cheating. By this I mean they call it poly which seems to make them feel entitled to having other partners NOW no matter what, and they are as inconsiderate to their partner as if they were having an affair.
They are honest that they would like to have other partners, but they don't get the ethical part so much, and that they need to treat existing partners with respect and caring, or to consider their needs, especially if both parties didn't agree upfront to an open relationship.
They don't get the multiple LOVING part, and instead take it as a chance to trample on their current partner(s) if it gets in the way of getting what they want with a new partner.
Worst of all to me, they then blame their current partners when things go wrong and they have the nerve to speak up about not wanting to be treated badly. If they were having an affair they'd just slink off to be with their new partner quietly. By saying that they are poly, they think that means they get to claim it's your fault, that what they want matters more than anything, and you are a bad person for raining on their parade. They don't seem to have a desire to recognize that they aren't willing to make the effort to give you dates, love, time and horribly in this case, to be a giving father. Your husband's choice of partner makes me sad too, I would feel sick if found out I was dating somebody who would encourage me to leave my husband and new baby at home while I went off to have fun instead of telling me to get my ass back there and help take care of them.
I really feel for you, it did not matter if you knew you were going to be OK with poly or not. His being selfish is the issue here as far as I can see. I know new parenthood can be terrifying but I feel sick at the maturity level he is displaying. Don't let him tell you this is your fault. Tell him you won't listen to that. I am sure there are many online resources for how to wrap your head around motherhood when the other partner goes absent, ones that might help you figure out how to ask him to take care of his child a few hours here and there so you can have the time and space you need to relax, sleep, wander around a department store, or meet with friends.
Happiness will never come to those who fail to appreciate what they already have.