Okay, I brought up the subject again.
What I did was telling him that I felt his "rule" was too vague, and that I would prefer if he selected on act he didn't want me to do with anyone else - regardless of whether I did it with him or not.
His reaction surprised me: he took a 180 turn and told me he thought he had been silly and that if I felt comfortable doing something, I should feel free to do it as long as I didn't force myself. And that he thought he was just being jealous at the idea that he might not be able to bring me enough, or that he wasn't good enough, and that somebody else would be better, and then I would want to leave him.
I reassured him that I didn't see that as possible, as what I do with one person or with another are separate things, and I wouldn't choose the person who does "more things", since the point isn't to choose one person but to have two strong trusting relationships. Or one big one with all three people I guess.
I am feeling much better about it, and we might want to discuss it again, mostly for his fears of not being good enough, that I feel won't just vanish, but I am much happier about the way this talk turned out.