Two things stand out to me.
First, I think you need to focus less on him when you're not with him. You are too preoccupied with what he's doing, who he's with, trying to keep track of who's who, how he manages his time, etc., while you need to engage more in your own life. Do you get out with friends, socialize, have hobbies? Invest in your career, your home, perhaps other potential beaus, and even just flirtations? Find something to be passionate about and throw yourself into it! It sounds like you have made him the central focus in your life, and that is why you get upset -- because you obviously cannot be at the center of his.
Second, you may always have a degree of discomfort or dissatisfaction in this situation if you are not into having more casual relationships. Given how many other partners he has, and the distance between you which naturally limits the time you can spend together, it would seem that this just will never be as committed and full-time as you would prefer. This doesn't make you flighty. It may mean, however, that ultimately you are not compatible in ways that are important to you.
Relationships shouldn't be angst-producing. If the scales are too often tipping in the direction of frustration and dissatisfaction instead of joy and fun, then maybe it's not the right relationship for you!