I'm not sure if this will help but it's what I did when first starting:
I had only ever been in monogamous relationships before but I agreed to enter into an open relationship - that later expanded to a polyamorous relationship - with my current partner.
Because I was so new to it I had a large amount of trouble knowing what the boundaries were and what I could or could not restrict, my partner had a similar problem, he never knew what was or what wasn't okay with me.
So, I came up with an idea. Seperately, we both came up with 3 lists.
1) A list of things we knew we could handle. (e.g. sex with other people)
2) A list of things we knew we couldn't handle. (e.g. one my boyfriend put forward - he didn't want me to be overly promiscuous, one I couldn't handle - him entering a relationship with someone he knew I really disliked)
3) A list of things we needed out of the relationship. (e.g. care, affection, trust, time etc.)
Doing this allowed us to compare our needs and figure out a way to compromise over boundaries.
This list was also designed to change with our relationship and, I found, that the more the things on the list 3 were being fulfilled, the less items there were on list 2.
Personally, I found that simply sitting together comparing lists and coming up with a new list of boundaries we were both comfortable with put me at ease almost at once. After doing this, the majority of my jealousy and worry faded.
However, it is possible that in doing this, you may discover that the two of you cannot come to a compromise and this probably means that you are not suited to one another. If this happens, it's unfortunate, but atleast both you and your partner will have a clear understanding of what you both want.