I think that there is less civil discrimination against poly than there was (or is) for homosexuality and BDSM culture, but the stigma attached to couples or singles who are "swingers" is thus far pretty negative. Of course, with the younger generation, I think that stigma may slowly begin to disintegrate.
I am concerned about the government determining that polyamorous people would not make good parents, because there are PLENTY of cases of divorced parents (which results in often worse situations), single parents, and just bad parents that aren't poly, but still screw kids over. I think this argument goes hand in hand with whether or not homosexual people are allowed to adopt or keep custody of their kids. Supporting, loving adults who put the needs of children above their own is the key.