Any advice on how I could talk to him about the possibility of me being polyamorous without hurting him or worrying him into insecurity?
What is it you want from partner?
a) to understand you are polyamorous and have the capacity to be in love with more than one person at a time.
b) to understand that you do not want to be exclusive and want to date other people too
c) Both A & B
d) something else?
Get clear on why you want to have this conversation and what you are hoping to get from having it.
As for how he may feel? That's for him to own. You have the conversation, he will feel whatever he feels, you will feel whatever you feel. Then go from there. You don't get to choose your feelings -- you DO get to choose your behavior.
You also never know how it will be til you actually have the conversation. You cannot be a mind reader.