I've really got nothing to add, I don't have the experience of being a swinger. But you did ask if anyone else "accidentally" became poly, and that's me. I wasn't looking for it, and I was actually anti-poly (for myself) when I found the right people. Our relationship progressed over the course of years from friendship to more like family...until finally an "official" decision was made to give poly a try. This was followed by lots and lots of research and lots and lots of mistakes, but we're still here.
Good luck, I'm rooting for you.