Norwester, you sound exactly like my wife. I can empathize with your husband considerably; when she first explained to me that she wanted to seriously pursue a relationship with another man, I was not interested in any other women and did not imagine that I had any prospects of any kind anyway. I struggled with the idea but I could see how much she needed to do this, and I had to start cramming down the fear and accepting that poly is who my wife is. Initially, the increased openness and communication in our marriage convinced me to accept the situation, but I am definitely less naturally poly than my wife, and it still isn't easy to deal sometimes.
Once I had basically accepted it, though, my own frame of mind was different, and I began to look at my own relationships with others differently. I started to realize that I have often held myself back from being fully present and myself when relating to others, in part out of fear that I might act "improperly" or overstep a boundary. I have not fallen in love with anyone else like my wife has, but I have realized that living in a poly relationship is a thing that some people are intrigued by. I haven't really "come out" to many people; for various reasons, it doesn't make sense. But in a fairly short period of time I have discovered that several relationships I had with other women are more nuanced than I had previously thought, and it is interesting to begin to dig in and explore where those relationships can go.
I think it will help your husband if you go out of your way for a while to show as much extra tenderness and care as you can. Try not to be too giddy and excited and filled with NRE in front of him; it can hurt even if you are genuinely happy for your partner. Make sure to spend some serious quality time reconnecting with him as soon as you can after seeing the new partner. And, as long as you are genuine in your intent to keep your husband primary, right now it's important for him to get a lot of reassurance on that. Try as much as you can to show him how much you appreciate being partnered with someone who is courageous enough to love you as you ACTUALLY ARE, rather than as a role that you aren't comfortable playing.
Hopefully he can see and appreciate the change in you as you become more open, accept yourself for who you are, and let go of the guilt. I'm pretty new on here, but I've recently been where your husband is at, and I'm happy to offer any support that I can if he thinks he might benefit from talking to somebody about it. Good luck.