Wow, I think you have to decide on one hand you have a limited open relationship and may have been based on something other than poly at the time. On the other you have 6 years invested in the relationship. Are there other aspects of your relationship other than this topic that are breaking down?
Tough choices all the way around. I see you trying to be open and honest but I guess not everyone can accept this lifestyle. I have had a rocky road myself where I was put out the door before I could ever even have the talk with a 3rd party in the secondary relationship, so I know it is difficult to just be shut out and not heard. A few years later, I am ok because of it but I am a lot more cynical than I used to be.
Guess I am not helping but I can definitely empathize. Some people just won't hear beyond their little window of the world.