I can only speak for myself, but I could never be in a marriage where my husband offered only black and white solutions to the issues you are describing. If you want to make any relationship work, poly, mono, or what have you, you have to be able to talk about how you're feeling without fear of an ultimatum.
That said, I also wouldn't stay silent if I were hurting as much as you seem to be, and just hope that other people figured it out on their own. I don't say that to be harsh at all, but sometimes people see what they want to see, or they see signs that you're unhappy and assume you'll come to them when you're ready. And sometimes that works out. But in your case, it seems like you held it in to the point where it had to come out in the form of this letter, and at that point it was maybe difficult for you to express yourself calmly? (That's conjecture on my part, based on being married to a man who at first bottled up everything.)
If you need time, ask for time. If they are honestly working to make you comfortable, they'll give you what you need to get that way.