BT, it sounds like you have reached a level of comfort and satisfaction in your relationship that makes you very happy. I could see how you could feel disappointment that he still wants to pursue an additional relationship and practice polyamory, because we so often want that sense of solidarity and the sharing of mutual goals with our loved ones. You are tuning in to a sense of oneness with him, and want to preserve it. I think that speaks volumes of the love you share with him. From what you've written, I think that's more what it is than jealousy per se.
But people develop and change and grow individually, so he may never reach that point of wanting only a monogamous situation, or he might feel that way some day, and who knows where your head will be then. I think that, if exercised with compassion and a respect for your feelings, he can still have another relationship without letting you feel like you're not getting enough of his love. It will take some work on both your parts. You may need to look at any insecurities you have and deal with them, remind yourself of all the wonderful things you are grateful for in your relationship, and be sure to express what needs to be communicated to him. He will have to be sensitive to your feelings, and may need to make special efforts to let you know how loved you are. Remember that his ability to be with, and even love, others does not mean he doesn't also experience a sense of oneness with you.
You both may want to read that book, The Five Love Languages, so you're both on the same page as far as knowing how to let the other be aware of what you feel. You may also need to sit down and discuss boundaries. My impression of how it's been thus far between you two, is that it's been fairly loose and casual as far as the swinging, threesomes, etc., but maybe you will feel better with a little more structure, such as a designated number of times per week he can go out and be with her. It's always nice when an established couple reserves a certain day each week for each other -- definitely make sure that you two share time together on "dates" and not just "running the household" type stuff.