I do find that the majority of poly people in my circle are gamers, but my wife won't date anyone that spends a great deal of their time on gaming, and we'd both much prefer hiking or some other physical pursuit. Outdoor hippie type would fit my wife and I well. I wouldn't call us anti-nerd/non-nerd though. I can be quite geeky when it comes to making good coffee or hot chocolate ;-)
I think your best bet is to go out and do whatever you enjoy doing, look for meetup.com groups that fit your interests, and also try inviting poly friendly friends. While they're a minority, there are some people in my poly circle that are more interested in regular exercise, gardening, and getting outside. Those are the people we're more apt to date, if we're not already.
ps: The less game oriented poly people I've met are people I met through my game oriented poly friends, along with local poly meetup groups. They're simply the outliers of the group.
Last edited by turtleHeart; 06-28-2012 at 04:31 AM.