So, you want a monogamous relationship, at least as far as him falling in love is concerned. He is communicating to you that this is not something that he can do. These two are mutually-incompatible.
If either of you "give in" and do what the other wants, there is going to be hostility and resentment.
I presume you made your marriage vows based on a monogamous basis to the relationship. A year and a half ago you agreed to trying the poly thing. (OK, this is overly-simplifying things, but I hope it makes the point) Now you have decided that this just doesn't work for you (which is absolutely your right).
To be blunt, I cannot see a way forward where the two of you stay together and both of you being happy. You have what seem to be incompatible "bottom-lines" where it comes to relationships.
"Listen, or your tongue will make you deaf." - Native American Proverb