You're entitled to make whatever rules/boundaries/guidelines or whatever else you want to call them in your own relationship. What does you wife say about your needs? Is she comfortable with them? If she is I fail to see a problem here at all. What works for one constellation may not work for another.
As for sex not being special if it's being shared with more than one person I assure you that it absolutely is. Usually the boundaries around rooms and shoes and whatnot come from a totally different place than from feeling a stonger attachement to those things than to a partner and what they do. Sometimes it's just something that has to be spelled out in poly because it's not a situation that's going to come up otherwise. For example I'm not comfortable with knowing that people are having sex in my house when I'm home. This doesn't just apply to my husband and his gf but to anyone but mostly with other people it's just not going to happen (or at the very least it would be considered rude).
You seem very angry. A lot of people are when they start out on this journey. A lot of the time it's growing pains to get to a new normal.
Everything will be ok in the end. If it's not ok it's not the end.