I came out to my family as poly only very recently. I kept lots of information from them for quite a few years to guard that part of my life, but once I realized that I was ready to face any worst-case-scenarios that I could come up with, I came out. I haven't had the experience yet of introducing my family to my Other Significant Other but that is a bridge to cross in the very near future.
At work I am presently in the closet because even though my company talks a good game about being tolerant and inclusive, I have seen and heard other people being ostracized for being outside the mainstream (for example, there is big talk lately about a transgendered woman who is working in another department.) People are still not as tolerant of different lifestyles as I'd like to think they are, and since this is the buckle of the Bible-belt, mainstream evangelical Baptist ways of thinking are in abundance.
I keep photos of both of my Significant Others on my desk, and even though I don't get questions about "who is that?" from anyone, I have already decided that I am not going to be misleading anymore. (A few months ago, I was at the local mall with both of my SOs, and bumped into someone from work. I introduced my OSO as "our best friend", and have felt rather sick about it ever since.) I'm looking forward to becoming the subject of office gossip in the near future, since I have decided that people knowing who the important people are in my life is much more important to me than hiding who I am, and who they are, and how important they are to me could ever be.
As far as strangers at restaurants go, I have never cared what strangers in any situation think.