I'm really new to this myself, so I'm sure some more experienced people will have their own insights to offer, but I'm pretty sure that polyamory is not supposed to be about "having it all." With any lifestyle choice (singleness, absolute monogamy, serial monogamy, polyamory, and everything else), there are things you gain and things you sacrifice. What we need to do, each of us, is understand ourselves and our own needs and desires -- what's essential, what's negotiable, and what's unacceptable -- and then be honest with ourselves and our partners about those things.
I'm dating a poly guy myself, and from that perspective I will absolutely echo MonoVCPHG's advice: take things s l o w l y. As someone who hadn't considered the poly lifestyle before meeting this particular guy, I know that I need to give myself a lot of time to process and pay attention to my feelings. Any kind of commitment or expectation for the future has to come only after I've had a lot of time to explore the territory and know how I feel about different aspects of it. I know it's tempting to go along with whatever you think will enable you to stay connected with this man, but you really need to be real about your needs and his needs, and whether you can actually come together in a lifestyle that works for both of you. This will take time, but better to wait and build a solid, real connection, than jump into something only to have all your hidden desires and unmet needs tear the relationship apart from the inside.