Tell him about what you have been feeling and what he can do to help alleviate your insecurities over it. I did this a few days ago, it was something really small, but it was negatively effecting me, so I mentioned that I was feeling really insecure about it and what he could do to help (again something really small). He said" I had no idea...". I knew he didn't and said as much and things changed. A simple little act stopped my turmoil and irrational fears.
His idea of a hobby and/or something indulgent to do that is just for you is something to seriously consider. This subject has come up in a number of places recently and it is something us moms have trouble doing, we need to be our own best primary. We spend so much time taking care of everyone else's needs, that ours get put by the wayside and forgoten. This can have serious negative effects that we don't even recognize because we are so busy worrying about everyone else.