Five years of being a secondary? That sucks! Ya, I would find someone else to fill the role he hasn't in that many years. If his wife moves back then you will have less of a relationship by the sounds of it. I think you are wise to prepare for the end. Metamour wives who are in dislike and struggling tend to "win" in the end in my experience. I would prepare for that too.
Stupid primary/secondary thing! Gah! Hate that shit. Love is love, to me there is no catagorizing it and controlling it. Take a look at some threads tagged "secondaries" "secondary" etc and see you are not alone.
Anyone want to be friends on Facebook?
Send me your name via PM