Having just "ventured out" earlier this year from trying to be monogamous for several years with my husband, I can't say that I had any jitters. It felt totally natural for me. My husband has had those jitters, though, and he often feels weird going with his new (girlfriend? not quite there yet) places without me. I'm thinking that he is a mono through and through, which is fine by me.
If the weirdness you're feeling is a sense of jealousy, that's something that needs to be dealt with before anything proceeds. My husband had a mini-breakdown about a week ago because I was spending too many nights with my bf. We had a heart to heart and he agreed to ALWAYS fill me in whenever he felt like he was being left out.
I know everyone says it, but openness is reallly the key to any relationship. Good luck to you guys!