I have a hard time explaining that to my wife. On some level, she finally gets it, but she still sees it as a sexual.
When I have to explain it, I just say that poly and swinging are both "open" or nonmonogamous. I see swinging and polyamory on a scale based on the emphasis. Swinging is more about the sex and polyamory is more about the love. But there is a lot of cross over.
The example I usually give is that polyamory just states that love is not a finite resource... so why limit love? So it is not so much about "doing" something but instead it is an expression of philosophy. If we are capable of loving many people, then the question can easily be reversed back to "Why choose monogamy?"