You are living through a fear that I have sometimes. I've struggled so hard to gain acceptance among those I care about, that if something were to merely "change" that it would be viewed as a failure. "Breaking up" or taking a break is a common direction of relationships, but when you've had to defend your life, it's viewed as an "I told you so" which I imagine is so frustrating for you.
Something my therapist told me a few months ago when I was struggling with the decision to be openly poly was to surround myself with people who loved and accepted me. There are many in my circle of family and friends who are intolerant of my life. There are an equal number who are indifferent and then there are the handful that I treasure. I've had to let go of some of the difficult relationships and cling to the ones that offered me the love and support I needed.
If you have people in your life that accept and support you, then you may need to lean on them a little more right now. You can seek out some poly groups in your area or reach out to people on here. I have found so much support and acceptance on the forums, it's on here that I have been able to find hope. There's always someone here who can understand what you're going through, that makes a big difference.
If your friends are pushing you to defend your life all the time, that must be very exhausting. I don't mean to sound harsh, but true and loyal friends make an effort to understand and support your choices. My closest friends were pretty freaked out initially about my husband and I being poly. Once the shock wore off, they became our biggest supporters. They had questions, sometimes they still do. But they merely want to understand and often just want to make sure they "have everything right" and not assume anything or offend us.
I could ramble a whole lot more, I'm still going through the "coming out" stuff with people. Feel free to message me anytime