OP, I think people in this thread are being a little harsh on you. Personally, I understand having affairs when you've exhausted the honest method. Plenty of people have affairs, and while it isn't right, it's in some ways a more socially valid option than poly. That is something to be aware of if you decide to embark on a poly life--you might be judged more harshly for being poly than for being unfaithful, actually. Affairs can be regarded as mistakes or slip-ups, and people are generally 'properly' ashamed of what they're doing in them--whereas when people see me out openly with my boyfriend while we're both wearing wedding rings and our spouses are at home, we clearly have no shame. That galls some people.
I think it sounds like your marriage isn't giving you what you need. Whether that's because you're poly or for some other reason isn't clear. I think either way you need to be honest with your husband about what you think you need--is it more sex? more variety? more emotional connections? people to spend time with when he's at work? a larger community to plug into for childrearing? Depending what you need, there may be other, even better solutions besides sex with other people.
Be honest with yourself first. What do you need? Then communicate it to your husband. Can he support you in that? If he can't, the relationship must end. And then you can decide what to do about poly from there. The real question IMO isn't whether you're poly; it's whether you can be partnered with this man.