The question you asked is a good one and is one that I've asked myself recently too. I actually even asked the question on here a month or so back.
Firstly, I want to applaud you for even getting to this stage in the way that you have. I've been poly for two years, with my girlfriend, and the first year was a huge learning curve of mistakes and bad behaviour. After that came the typical strict-ish guidelines, along with complete disclosure of everything. Finally, now, we have reached a stage of trust and relaxed. You seem way ahead of the game.
It's normal to feel what you're feeling. Being poly is kind of like being at University. You can study for years, but when that big test comes, you're as nervous as hell.
I completely empathise with being unsure of what to ask, what to know. In the early days, it helped me to know everything. These days, I don't want to know anything at all. What you choose is definitely down to you.
I have noticed one thing. When something has just happened, like a date, knowing feels like a truck has hit me. But, a few weeks or months later, knowing something isn't as bad.
I don't believe in 'don't ask don't tell' - the option of an open book is good, in my opinion. But what you have to ask yourself is: "how does my mind work?" If you hear those words "yes, we had sex", are you going to imagine all kinds of things? Often, you hear something unexpected. He could say "we didn't go all the way". What are you going to imagine then?
Ultimately, it comes down to whether or not you feel them doing something would effect your relationship. You are very, very, very lucky that he is still being so wonderful to you. Read the pages here and you will see so many people upset because their partners are neglecting them whilst they chase NRE. From the sounds of it, your relationship is solid. Nothing promotes love and security more than personal freedom, in my opinion.
Poly was something that I believed in, but was quite painful for me, when I knew details. Now that I don't ask, it's easier. I know my GF is dating; I know who she's dating; I know when. I know if she meets someone new, if she's getting to know them. The rest is up to her. If it doesn't effect me, I needn't be bothered. And you know.... knowing doesn't stop anything bad from happening. I have felt neglected or insecure in my relationship at times, even when I've known what's going on. But reminding myself that it's her body, her heart, her mind... being more autonomous... has truly helped me.
My advice would be to follow your instincts. You'll probably be overcome with curiosity and ask him, when the time feels right. I'd say give it a few days, at least. And if you do ask him and if the knowledge hits you like a truck, you'll probably cry, you'll go to bed, you'll wake up, you'll see him, you'll feel his love... and you'll know that, actually, everything's going to be alright.