"One thing missing these days is physical intimacy."
God that sucks, well I think it does anyway. Maybe you're not getting "physically rejected" per se, but you're getting rejected by way of that lack of trust. That actually seems worse than if it was because "I'm not attracted anymore." It's like being rejected for a personality reason. Ouch, ouch.
I don't suppose there's any chance he could give you a timetable on this aspect of it. Naming a time (estimated date) when he thinks he will be able to do the physical intimacy thing again. You would think he must miss it too, even if it doesn't feel safe to him right now.
God that sucks.