So, the only woman you have an issue with is the local woman he sees once a week. That's understandable. But you need to understand this other woman IS important to him; it's not that she just happens to be convenient. He's poly, there are plenty of other women he could date, he chooses to date her because he has feelings for her. And, yes, because she's close, she's able to give him more....and one day he may end up giving her more, as she gives him more....and he may not have time for you. That's how relationships build. Love takes more than emotion....it takes that, yes, but it also requires time and effort.
That's part of the consequences of distance.
You need to be happy for him, in his other relationships, because you care for him. Otherwise, you're always going to be unhappy and worried.
Just know that you are secure in yourself and your relationships, with or without him