Some people don't like the primary/secondary labels because they can be misleading. The most common use is to point out who you live with or spend the most time with. But some people take it as a measure of the love. So he may be avoiding that to avoid confusion.
People talk about each other. People seek advise from others. So I don't think you did anything wrong. But people do worry about being talked about in a negative manner. One suggestion is to get everyone together and talk about this issue and see how everyone feels about it. This is one of those good issues that you helps develop communication skills in the group (if done well).