Communication, communication, communication!! When people ask me what makes this lifestyle work so well for me I tell them that the girls and I have NO secrets whatsoever. When one of them is feeling down about something we have to talk about it. Communication leads to some not so pleasant conversations, but they are necessary ones.
What you are feeling is VERY real and most importantly, OK to feel. as you said, you're not poly so it's tough to understand how that works for those of us that are. That's where talking comes in. Sit with your bf and express your feelings to him. Let him know that you suffer from comparison disease. There were a lot of conversations that were had at the start of this lifestyle where my wife expressed a lot of those concerns. Through talking and letting her know exactly where my head and heart were at she came to understand how it was possible for me to love someone else as much as I loved her and how that didn't diminish our relationship in any way.
Also, this lifestyle is WORK!! No doubt about it, but I think that the works absolutely is worth it once you find a relationship that works best for you and your partner(s). Good luck and please use the resources (other people) here to help you out in your time of need.