My husband and I have been together since I was 17. Back then the only example of polyamory that we saw was done excessively badly so we never would have applied that label to ourselves at the time. However we have always had our own agreements about what was "allowed" with others. It has changed and morphed over the years.
My advice to you would be to start out in a place that you are both comfortable. You don't need to have a full blown second relationship right off the bat. If for the time being what's comfortable is flirting and fantasizing about other relationships then do that. Keep talking about how you feel and talk about what step you can take next that you are comfortable with.
It takes some time to overcome how we are taught that we "should" feel in a relationship if our partner desires someone else. You say that you feel jealous but you don't want to. Chances are that a lot of that stems from being told, both overtly and covertly, over the years that you should be jealous.
There really isn't any rush. Enjoy the ride. Take things as they come and keep talking and being honest about how you feel.
Everything will be ok in the end. If it's not ok it's not the end.