I don't have any concrete suggestions but I can empathize fully. For me, I know that just being around people (other than my boys) and being out of the house are both draining and that I need both preparation and recovery time. MrS knows this and Dude is learning.
For instance, in the morning I know I need an hour of reading/drinking coffee/lounging to "wake up" and prepare for the the day. When I get home from work I need another hour to "unwind" - change into lounging clothes, have a beer, check these forums, etc. During these time I don't mind that they are here but I don't want to talk/touch/engage with them. Dude was always trying to hug/ engage me right when I walked in the door and I would get all irritable. Now he knows to ask "are you 'home' yet?"
Also, for me, I don't like doing social / out of the house / being around people things on the spur of the moment - it's like I need time to "garner my resources". For example - The other day I get home from work, do my "unwinding" and am looking forward to a lazy evening. Dude wants to go out to a restaurant for dinner (MrS is out of town). I do NOT - I have just been at work all day - and leaving the house to go to a place that contains people just sounds exhausting. With MrS this would not be a problem - he is fine with going to a restaurant by himself, but Dude wants to go to the restaurant with me
(because he knows I would enjoy the food there and wants to share it with me). So he counters with the suggestion that we go the next day - when I only have 1/2 day of work and can come home for an afternoon at home before going to the restaurant. So, now I have a day to prepare mentally AND a plan for recharging beforehand - I agree.
I try not to agree to plans that don't allow me adequate preparation / recharging time in the first place. So I rarely have to back out at the last minute. The boys have learned/are learning to accommodate this - and will just make plans with each other to go do stuff, with an open invitation for me to join them but without the expectation that I will.