I think my exposure came from Three's Company and Star Wars. They both had a love triangle(ish) subplot. (Though Star Wars went icky as the story progressed.) Everyone talked about Jack or Leis having to choose who they wanted to be with. I thought that was unfair. After all, they were all friends, why not let them all stay together. I didn't want the adventure/friendship to end.
I also read a lot of sci-fi (but didn't read Heinlein until long after I identified as poly). Lots of sci-fi has poly oriented material.
Most women I dated were bisexual and being a typical guy, I wanted to help them explore their bisexual side (and watch/join in). That idea led me to consider swinging, but I wanted it to be more of a relationship. After a bit, I fially discovered polyamory on the internet and I realized that described me perfectly.