Only way to change that would be to move to an area that has more of your vibe. Boston, Austin, and Seattle are known for their poly populations. But moving of course is a big deal, not something, perhaps, that you'd want to do.
Any poly groups in your area? Perhaps if you google "Vermont polyamory" something will turn up ...
Love means never having to say, "Put down that meat cleaver!"