Welcome to our forum.
I think it's very common for people to struggle with the way they were brought up. Society as a whole certainly doesn't support polyamory (heck, as a nation we're still struggling with same-sex marriage issues). You have to figure that when you start to have some polyamorous feelings, it's going to come with some anxiety.
I believe it's something of a fallacy to worry that if you start loving one person more, you will have to start loving another person less. Love isn't like a pie that runs out, it's more like sunshine, there's plenty for everyone. If you should get involved with another woman sometime, for example, there's no reason why you can't keep loving your husband just as much. You just love different people in different ways.
Of course, you do have to be cautious about NRE (New Relationship Energy). Sometimes the new person seems so exciting that you can accidentally neglect your relationship with the pre-existing person. So if you do get involved in a new relationship, do be extra careful to give your husband plenty of attention and reassurance.
Hope you find Polyamory.com to be helpful as I have.
Love means never having to say, "Put down that meat cleaver!"