I have to agree with Inyourendo in that I'm much happier outside of my 17-year marriage now that I'm not feeling derided for who I am and constantly being told (indirectly or directly) that what I liked and wanted was stupid or selfish.
However...
I do not believe that just because you're unhappy and trying to figure out what you want now, it means that you need to dissolve the relationship because it's never going to work.
Hell, if I'd outlined all the problems I had in the beginning of my relationship, I think a number of folks would have been appalled. We all were new at this and stumbled our way through like drunken elephants in a corn maze.
I was envious of the time P and M1 shared (especially when he was unemployed and they could spend all day doing whatever, while I was working OT at the time). I didn't like seeing their public affection displayed in front of me. I didn't want to see their gushy Facebook back-and-forth stuff. Et cetera.
It took time and work to get through these things. I asked M1 to exclude me from seeing her gooshier FB posts to P (I needed the option to "leave the room" as it were, since FB isn't all that good at it). We agreed to "family friendly" levels of affection when we're all together. Our time together is much more balanced now. It didn't happen overnight. It happened with a lot of self-introspection ("What do I need? What's bothering me really?"), a lot of talking ("I know I'm bothered by the FB thing for <x> reasons, and I don't want to hide or defriend you - how can we find a happy medium?"), and mutual respect and the desire to come to a common middle ground.
(I realize that my feelings are my own, and I was prepared for M1 to say "my FB is mine - suck it up", in which case I would have hidden her from my feed and said that's that. I was *prepared* to do what I needed to do, but wanted to see if we could find a happy medium instead, and we did.)
None of us ended up steamrolled (although in our blundering, some of us did feel that way at times).
And yes, we STILL have issues. But we've gotten better at getting them out in the open, doing the self-introspection, and talking - it's kind of becoming old hat at this point.
So no, I don't believe your discomfort necessarily means that you need to give up. I find that defeatist ("It's never going to work so why try?") and I hate seeing people give up on something they never even had a chance to work at, or didn't feel that they could because they didn't have the confidence in themselves.
I do think it's going to take some earnest work on your part, along with your partner, in order to get through the hard stuff. If his OSO is so inclined, then it could very well help to get her involved too, but if she feels you two need to work out your own problems, that's also valid. Start inside and work out - starting with yourself (self-introspection, what do you NEED out of a relationship, what do you WANT, what compromises would work for you so that you still feel you're getting what you need?), then with your partner, then maybe with his OSO if she's willing.
I would caution you that it might take multiple go-rounds to get through an issue. P and I felt as though we were hitting the same roadblocks over and over and over again until we finally peeled back that onion one step at a time and got through the surface stuff to the root of the issue.
If you find that, after trying for some period of time, it's still not working, that no progress has been made, and you're feeling exactly the same as you were before, then yes, it might be time to think about moving on.
Only you can decide if it's worth the work. And only you can decide if the workload is too high, or if it's manageable. But if you and he both decide that it's worth it, and that it's doable, then go for it.
Edited to add: Whoops... I forgot that you are also in a relationship with B. Doesn't change my post much except to say that you probably should all talk, then, rather than leaving B out of the loop at first, since there's a direct relationship there as well.