Does your husband actually know any of these thoughts that you're having? Have you considered communicating any of this to him before proceeding?
Since you haven't provided much detail of what's wrong with your marriage, there's not a lot to go on. I'd say more than anything it looks like you're comparing something new and shiny to something worn and used. Never a good idea since eventually all things lose the luster.
I'd suggest taking a step back from the new relationship and spend some time focusing on improving the one with your husband, if you decide you do want to try. You won't have much urge to do anything in the old relationship while your head is spinning with thoughts of how great the new guy is. NRE can be toxic, and kill relationships that otherwise were healthy or at least decent.
It's your life though, and only you can decide what to do with your situation.