I always find it's better to be upfront and explicit. Beating around the bush by saying you don't mind if he shags other women is not the same as telling him how you feel and what your needs are.
To break the ice, you might download a video about a polyamorous family. Watch it together and see what he thinks of it.
After that, my way would be to just jump in and spill it. But then, I'm terrible at keeping secrets. Even if that's not your MO, I think the longer you keep it in, the more and more difficult it will be to tell him.
I mean, what's the worst that can happen? He won't like it, he'll tell you so, and you'll talk about it. At best, he'll feel the same way (maybe laughing it off was because he thought you were testing him?) and you can start to explore.
I totally get the whole "he wants to make me happy, but I don't want to make him unhappy" thing. That's Gralson all the way. He always puts other people first (if he likes them) and then begrudges it later. It took me years to figure out how deep that runs, and then to figure out when he's saying yes but not really wanting it
“As I am sure any cat owner will be able to tell you,
someone else putting you in a box is entirely different
from getting into a box yourself.” —bisexualbaker