I didn't call him a babysitter. I was describing the service provided. And I think that his effort and commitment are higher than most in his age group....actually a compliment. I'll do a quick poll of young men and you do the same and see what's more typical.
My point is if this bothers him enough to ask he should know the truth as unpleasant as it might be for him to hear....so he can decide how he wants to precede. As you said your husband had to hear the same thing and is fine.