This is obviously eating you up and it will ruin your relationship if you don't do something about it. You know already that he could have already moved on to another person and he's not so much polyamorous as polysexual. I think you need to have a sit down talk with the two of them and say exactly what you need to say and get the answers you need. Yes, lots of tears one way or another but that's the way it's going to be. I'm sorry you're going through a bad spot in your life.
It's really possible he's polyamorous, and if that's the case he needs to know he needs to pay lots more attention to you so you get your needs met even he is infatuated with her for the time being. Could be you need to split up for a while and sort things out.
I know this isn't what you wanted to hear, but IME it has to happen.