Welcome to our forum.
It sounds like it's all about how serious your boyfriend's other relationships are. If they're casual relationships, then there is no threat at all. If they're serious relationships, then maybe they'll grow and grow until they take over your relationship.
It sounds like it's very important that all of your boyfriend's other partners be secondary partners. If he had another primary partner in addition to you, you would feel very threatened.
You have to work through these insecurities, and try to have as much faith in your boyfriend as you seem to. If you were 100% sure that he'd never leave you, why would have these fears of being replaced? You must not be 100% sure.
Are the two of you still seeing a counselor? Perhaps you still need some counseling. Try to find a counselor who is poly-friendly, of course.
I hope you can get things worked out, and that Polyamory.com can be some help.
Love means never having to say, "Put down that meat cleaver!"