Welcome to the forum!
Your kind of situation is not an easy one, although I will say that if you are really feeling the urge to explore polyamory (or rather nonmonogamy) with your boyfriend, it is not a good idea to wait until you are married. If this is something that is going to end up being a dealbreaker for one or the other of you on either end, you don't want to have to go through the hassle of a divorce in order to go your separate ways.
The best thing you can really do is talk to him about it. It might be difficult, and he might not react the way you want him to, but he cannot process the information or give any thought to it unless you tell him up front and honestly what it is you are looking for. You have told him you are okay with him having sexual interactions with other females, but it sounds like you have not expressed that you really would like that freedom to explore as well.
There are a number of books on the subject available to buy or to check out from your local library. Opening Up by Tristan Taormino and The Ethical Slut are two of the more popular ones out there. If you two absolutely want to avoid emotional connections with more than one person, you may want to look into the swinging community. Swinging is very different from what most people would consider polyamory, but the community is set up to help keep people from falling in love.
I will also put out a red flag for you right now. If you are interested in keeping all outside encounters strictly physical, then you need to avoid the people that give you the butterflies. You already have some emotional/romantic feelings for these people, and those will probably be intensified if you get involved in a physical relationship with these people. If you do want to explore or pursue people in your social group or who give you butterflies, then it is a good idea to talk to your boyfriend about what the two of you will do if love becomes a thing with another person.
One of the most important things to keep in mind right now is that you need to do what is right for you. If monogamy isn't right for you, then you need to address that issue NOW before you go any further in this relationship. You need to give him a chance to process what you are telling him and to think about whether that is a world he can share with you or not. The possibility of losing him may be daunting, but it is far less scary than building a life together for years and years in a marriage and then having all this spring back up in your face. The earlier you get things like this sorted out the better. Do your best not to avoid looking at it anymore.
If he is willing to talk to you about it or explore the issue, many states and cities have local poly groups that meet up regularly and these places can be a great resource to learn from people who are actually living the life. My fiance, my boyfriend, and I have found the local group to be very helpful for finding other people to relate to and ask advice from.
I would also say that if your relationship is rocky or has other issues at the moment, that those do need to be addressed, open or not. Relationship counseling is usually a really good thing for couples to learn how to work together and break down each other's defenses. If you are having problems relating to each other, then these need to be resolved whether you are mono or poly.
I wish you the best as you start to explore this facet of your personality and start to figure out if it is something that will become a part of your identity or not. Face this time in your life with courage. It will not be easy, but you will probably mature a lot in the process and come out on the other side much more equipped to deal with other difficult life situations. Feel free to PM me or respond in this topic if you have any more questions or just want to talk. We are about the same age and I know in my hometown it is hard to find young poly people, so I'm willing to be an ear to listen if you would like. :-)