In my mind, for something to be immoral, it has to be having a negative effect of hurting someone. Polyamory as an idea, and often in practice, is a completely ethical relationship style that makes an even larger number of people happy because there are usually more people involved than in a monogomous relationship. To me polyamory means multiple relationships (or the potential to have such) with the knowledge and consent of all involved. In my experience, when this knowledge and consent is reached, it usually leads to happy and fulfilling relationships. Its true that there can be drama, heartbreak, and negative emotions in poly relationships, even when they are being conducted in a healthy ethical matter. That said, there can be those negative emotions in an ethical monogomous relationship as well. Life is full of pleasure and pain, and they will come and go in any situation. That can't be used to argue that polyamory is immoral though, any more than it can be used to argue that monogomy is immoral. It is a lifestyle choice, and as long as all involved happily make a choice to be involved in that kind of lifestyle, it is not immoral by my standards.
His standards for immorality might be different. They may involve some religious structure, or may involve ideals of ownership and possessiveness in a relationship, or a good many other things. By his individual morality polyamory may be wrong, even if it is not by your moral standards. The only thing to do is understand that you may have different morals and accept that. And try to encourage him to understand that you must live by your moral standards, not anyone else's.