He may be very much in love with your wife and, simultaneously, very insecure. If his power over his own wife is one of the ways he confirms his self-worth, you've got yourself a helluva conundrum... and I feel for you. It's not impossible to get beyond, but... oh my gosh... it's slow. Your wife will not serve the same function as his own, and to change the dynamics in his own marriage to allow room for it would be... threatening, to say the least. And to say you and your wife are not a threat to their marriage may be absolutely true, but it takes time to understand and believe.
I do feel bad for you guys, but you know what? With that little bit of insight you've provided, I feel bad for him. If he really loves your wife, and he's not totally blind, then he probably knows he's being a jerk and doesn't know how to fix it or really, himself.
My advice: do you love them? Give them time. There's no reason not to, unless it hurts too bad.
In our poly relationship, I have been so frustrated by this dynamic. It took years of my husband helping me look through eyes other than my own for me to finally think for myself, in relation to my OSO, "You don't scare me, and I'm not running from you. If you don't want me, you're gonna have to push me away, because I'm not going ANYWHERE."