That sounds like an emotionally unpredictable and stressful situation. It also sounds like you share something really intense with him, especially after having such a long break and then coming back together and it still being just as wonderful.
All of that struggling and compromising on your part must be really unrewarding at this point, given then he's not matching you with the compromise and care (from my perspective). You sound like a giving, understanding person really looking to make a strong connection, and like you are starting to realize that this might not be it.
Your heart is your best guide; you know what is okay with you, and what is not okay with you. It doesn't sound like there is a lot of wiggle room for you to get what you want with this man, and you certainly deserve to get what you want in life.
I think that you've been really brave - opening your life, heart and body up to a man. The fact that he is your "first" in a lot of ways probably makes him even more special, on top of all of the history and experiences that you have had with him.
Be honest. Stand up for yourself. If he still can't give you what you want, then you have an answer that will probably hurt a LOT, but at least you'll know that this is not the man for you.
I have to say too, that I have a lot of respect for you stepping away from the situation when you knew that it wasn't healthy. That's a really good sign that you are in tune with yourself, and aren't afraid to set good boundaries.