I have to say, I'm really pleased with all the responses I've seen here. I'm grateful for all the input. It's really helping me Sort things out in my head.
Sparklepop hit it right not the head. I just want to be respected by my wife's "others". I want them to acknowledge that I am there and to respect our relationship. Being married is a huge, important (the most important actually) part of my life. I've cut family out of my life for not respecting that. (that's a whole other thing that doesn't involve poly as much as an entire side of my family being racist pigs)
This guy has now recontacted her. I've expressed the fact that I have a base gut feeling about him that says this will be trouble, but because it is just gut instinct I have no tangible reason to say no. She explained to me that she would like to meet with him again, and explain how poly works for us.
So they are actually on that lunch date right now. While I am still not thrilled my wife is making an effort to get things with this guy to a point where I might be comfortable. We shall see where things lead. I'm a lot more comfortable having had time to bounce things around on here and try to sort things out in my head.