Sounds like you could benefit from resources.
That one in particular contains links to going from mono to poly and avoiding pitfalls.
You have just started to really talk to each other. Keep talking and figure out your boundaries and HOW you want to approach this. Don't be in a big rush to explore dating. Talk about how you want to break up too, even if you do not want to or expect to. You have a kid to provide for here. Maybe consider postnuptial agreements. Who knows.
But if you can't talk about it honestly now
while nothing is happening and it is just the two of you? If adding a new person in your life (baby) caused this much hooha? What will adding yet another bring? More hooha?
Think about opening "too fast" and adding yet another person who changes your family dynamic and familymath/polymath
there. Think about opening "just right speed" so both of you can weather the transition time realistically.
Maybe you prefer to sort that "what if we break up?" stuff all out before Opening. Because you DO NOT want to be sorting all that out if you get to some kind of crisis mode after having Opened.
Better to have a plan and not need it than need it and be all crazy. YOU are responsible for your own well being. As for the mom thing -- even as a SAHM, I put my kid in daycare twice a week part time. I swapped sitting with another mom and housekeeping too.
To free me up to volunteer and interact with adults or have "me time" to read and garden for my OWN mental health and well being. Again, you are responsible for your own well being.