Well, sounds like you are trying to peel the layers and understand each other then. That's a plus!
New house and baby on the stress scale
are weighty. Ditto adding an OSO, though what you would rate it since it isn't an obvious one.
I don't know what other things you have going on there, but no being on the same page as to conception -- that's a major. Gotta get that sorted.
Figure out the other stressy things in your life and see where you are at. Some life things we have no control over. But when when we choose to be Open or Closed -- based on what ELSE is going on in our lives -- that we do have control over. Consider carefully when is the Right Time.
As for any other concerns he has, he is unable to voice them. He just keeps saying that he isn't comfortable with it. To me that speaks to a worry that I will be stolen away or a general dislike to the idea of me having sex with another man. Why that would be, I don't know. Maybe he is worried I'll enjoy them more? Just pure male pride? All is conjecture since he hasn't been able to describe why he is uncomfortable with it.
Maybe it is also in part speaking to discomfort in doing the work of introspection? Calling himself into account? Calling you into account?
If he needs a break before coming back to this discussion -- see if you can talk about THAT?
His need for breaks, and your need for check ins. Once a month? Once a week? Not all elephant problems can be broken down in a single session. People get tired. Need to eat, sleep, work, pee, have fun, etc.
Take it one step at a time, but do take baby steps to keep moving it forward. Dunno if there's anything here you guys can use in your talks.