Are you of compatible sexualities to be in a triad? That is, are you both male, or is at least one of you bisexual if you're an opposite-sex couple?
From what you're saying, it seems to me the worst part for you would be feeling excluded and rejected. Therefore I would suggest trying to get close to any partner he might meet. Become friends with them, get to know them. Even if you don't end up being involved sexually, emotional connections would be important, as it's less threatening to know "he's out with so and so" than "he's out with this person I don't know and who I'm therefore picturing as threatening to me".
He doesn't want to leave you. He wants to be with you. He is being honest and letting you know that he might fall in love while still loving you. He's also letting you know that he wouldn't feel true to himself if he ignored one of the people he loves.
It's not choosing, to a poly person. Choosing would mean being with only you, or only the other person. That's the way it feels for him, I'm sure. To you, it probably seems like anything short of rejecting anyone who isn't you is choosing them over you. But that's not the way it feels for him and it doesn't mean he loves you any less. It's probably hard to understand on an emotional level, but if you manage to, you should feel better.
Make sure to be honest with him about your feelings. It's one thing to be willing to sacrifice in order to stay with him, it's another to keep quiet about the efforts you're making. He does need to know if it is hard for you, he can do his best to make it less hard. He could seriously hurt you by accident, simply for not knowing how you feel. He doesn't understand how you work anymore that you understand how he does. He knows the basics on an intellectual level but won't relate, won't realise he might be hurting you. Be very open about it, without being reproachful.
Just let him know "I feel this way, I want to stop feeling this way, let's work on it together".
I wish you the best of luck, feel free to come back often for advice on specific situations as they arise.