I am sorry your friend did not think he could accept polyamory. It does seem to have let you understand more about yourself.
I think many people are more comfortable with monogamy with a side of adultery than with straight out polyamory. Probably it is because one is common and familiar as seen in many movies, TV shows and books. However, polyamory is still new enough that it has to be explained. Heinlein is a rare exception to this.