I'm totally new to this, in fact I'm going on my first 'date' with a buddy of mine tonight...
What I would say is, primary is primary - if that's the nature of your arrangement, primary always comes first. That being said, he's an adult, I'd just ask him. It shouldn't be difficult for him to give an honest answer. If you're concerned about him, let it be his call.