It sounds like he is either done with you or has new relationship energy (NRE). In light of the fact that he had a party and invited his other lovers (how many does he have if can make a party out of them?!!!) I would say he wants you to invite this woman into his fold willingly and without emotions other than compersion. Perhaps he has the idea that he would like a harem?
It sounds like he might be frustrated that you would feel anything else but compersion. That is disrespectful and unfair... not to mention uncaring and unloving. It is totally okay to be hurt and need some attention towards your concerns and feelings of neglect/abandonment; whatever it is you feel.
That is just my take on it though, I know some poly circles would not think so as they believe that a person has a right to their freedom and that they can date whomever regardless of any other partners feelings. The belief seems to be that people should have good self esteems and feelings of self worth and should not need help or support working on that... perhaps he is like that? I would think you would know after three years.
I haven't known any relationship to last very long based on the kind of emotional connection and depth that comes with believing that jealousy and any other hard emotions are to be avoided. It seems that you once had a connection and it has faded.
Anyone want to be friends on Facebook?
Send me your name via PM