If you use polyamory to mean any type of love, whether it's Agápe, Éros, Philia, or Storge (to use the Greek model) - in other words if you include the love of family, pets and possessions, then it devalues the term to make it completely useless, in my opinion.
Most people in the poly community define it as a romantic love of some sort, which often includes sexual aspects. And it is this love type which distinguishes polyamory from monogamy, which makes it useful as a descriptive term.
The problem with the perennially-occurring "we are all poly" statement is that it usually only serves to alienate and upset folks who don't feel the same way about romantic love and who feel that, by being put in the same pot, they are being told that it's somehow unnatural to only want one person romantically in their lives.
"Listen, or your tongue will make you deaf." - Native American Proverb