I know quite a few people who are only involved with one person at a time, sexually, emotionally and loving. They call themselves monogamous for that reason. They don't want to be involved with more than one person at a time, either through choice or wiring (and for the sake of this argument I would suggest that it doesn't matter which).
So no, I don't think that we are all polyamorous at all.
"Listen, or your tongue will make you deaf." - Native American Proverb