Hello there and welcome.
I would agree with Mags that your boyfriend's lack of emotional expression is a good place to start. Poly couples need to have a good foundation of honesty, open communication, respect and consideration or empathy for everyone involved before starting out. Being emotional is a huge part of being able to express oneself and be able to achieve a foundation... there are other views on what a solid foundation is, the one I have expressed is mine. If you want to know more about other ideas, you can do a tag search for "foundations."
You could also look at some information on what others have learned about poly who have experienced relationship dynamics that way by doing a search for "lessons" in the tags.
It sounds to me that there is some disconnect for you and your boyfriend. Having a child will do that most of the time and you have a young one. Rather than finding connection elsewhere I think it might be a good place to start with connecting again with your bf.
Maybe finding a babysitter and going on a date once a week, or at the very least, when the baby is in bed, settling in for some one on one time. Express to him your concern for not knowing his feelings and ask if you can work on that together. Tell him that you don't feel connected to him and that it worries you and that you want to make every attempt to establish some of that old feelings you had of when you started out dating. They are not necessarily lost, just buried under responsibility and life creeping in. Suggest that you find them again... maybe going to therapy to help him learn how to express himself will help.
A lot of men have been trained not to show emotions by their parents.... I'm a parent too, its so important to make sure that we teach our kids to be emotionally strong, especially boys. What he does for himself now will rub off on his child for the better too.
As to poly and this other man, I think for now I would consider his presence in your life a gift. He has taught you that there is trouble in your relationship and it needs fixing. Getting a new man is likely not going to fix that. If your bf is not willing to do any work on your relationship after you have told him that you are concerned and suggest some ideas that you and he could work on, then I think I would move on. Adding someone and asking for a poly relationship under the circumstances you are in can often be disastrous and hurtful to already existing partners.
Then again, if he doesn't open his eyes to what you are saying the first go around then asking to open your relationship might make him sit up and listen. I think it better to opt for asking him if you can work on your relationship first though. Whatever you do... please please don't cheat. If you need any help with a decision on cheating vs not cheating, just do a tag search on "cheating" and I suspect you will be convinced not to in no time.