Do you want to be in a relationship with A, a relationship with A and B, or just to know where you stand?
In my opinion talking to A would be your best first step once you've worked out exactly what you want. If it's options 1 or 3 you're after, there's no real reason to involve B in the conversation at this stage. If you're after option 2 and it turns out that A is interested in the idea then you'd need to talk to B, but there's no point otherwise.
Keep in mind that A may only be interested in a FWB or FB relationship and the answer could be "No". By the sound of it he's never given you any promises of more. Whatever you do, don't agree to something you don't want in the hope of changing the rules later on.