If the relationship isn't going well, breaking up is an option.
With the pregnancy, it's possible that it would work better for you not to have other relationships right now, and to ask for the support of your husband and his girlfriend, as well as your friends and family, etc. You might need to focus on yourself and your baby, sometimes that's the way it goes.
I suggest you think about it, talk about it with your husband, and evaluate the relationship with your boyfriend. If it isn't going well, you shouldn't hang on to it. Maybe you do because you are pregnant and want more company. Maybe you do because your husband has someone and you want someone too. Make sure you are with your boyfriend because you want to be with him, and if that's not the fact, consider ending things.
On the other hand, how is he "rubbing you the wrong way"? Is it fixable? I would be interested in more details.