Secondary means exactly that. Your relationship is the second priority in the relationship. His wife and family come first ultimately. If you understand and accept that, then you can truly enjoy your poly relationship. Where your stuff is stored or even if you co-habitate have little bearing on being a secondary.
Yes, there is a pecking order and yes you will be considered second in that to his primary.
Yes, you generally will spend less time with him than his primary.
This doesn't mean you aren't an important and loved part of the relationship. Just don't get all possessive and selfish towards his time.
Do make sure you feel healthy and taken care of. Are you looking for a primary as well or other secondaries? That can be a good way to balance your own attention to him.
Mono "Professional Life Committed Secondary"