Oh sweetie, you are not alone. From what you have said it sounds like your husband has some major control issues that manifest whenever the situation doesn't go as he would like. Saying those things to you is not loving. It is transparent manipulation on several levels. He wants to control your feelings, but also your actions. He cannot control the first, and he can only control the second if you let him.
To be fair, his lashing out sounds like a lot of fear and insecurity - and he is most definitely deflecting those feelings towards you. If he wants to have a lasting relationship with you then he needs to find another way to express his worries. Threatening you all of the time leads to an almost certain outcome - eventually it just isn't a threat anymore because the constant state of "will he leave me" is going to drive you mad. Being left is probably what HE fears most, but he cannot face that.
There is a lot going on here - you need to spend some time with people who remind you that you are worthy and lovable. Your feelings are valid and not subject to anyone's approval! If you are craving contact with this other man (and no wonder) it is probably because he makes you feel cherished. There is nothing wrong with that! We all deserve it.
When you are ready (and I hesitate to give this advice, but keep it in mind) you need to diffuse these threats your husband is making. Next time he says, "decide in 30 seconds if we are still married" you need to point it back at him and remind him that you are being honest about your feelings throughout this process. He needs to accept you as you are - so HE NEEDS TO DECIDE, not you. You have already decided to be with him, why is he still looking for validation of that unless he fears that you will do the leaving? Remind him that you are with him by CHOICE, not because you can't make it on your own. And of COURSE you can make it on your own. It would not be easy, but don't (PLEASE PLEASE) give him that kind of power over you. That is crossing into a very unhealthy area for you both.
I hope things get better for you. We are here for you any time!