Ok, so there is you, partner A (accepting of affection and overall happy with you), and partner B (doesn't accept affection and refuses to say "I love you" now).
Partner A and Partner B decide to add you to their relationship. They decide that it is either all of you, or you get kicked out so they can return to the way they were before. However, the way they were before was that Partner A was not very happy.
Partner B decides he doesn't want to be with you anymore. Or if he does, he sucks at showing you, to the point that you are unhappy in the relationship. You agree to work on it, but feel that it is unfair that you are treated so poorly. Partner A agrees that it is unfair and thinks he could be happy with you, but couldn't be happy going back to being just with Partner B.
Now, I understand that Partners A and B entered into this together, but does that mean they have to leave it together? I don't think so. If Partner A isn't happy, he needs to figure out what would make him happy and go for it. If it means breaking a promise to Partner B, so be it. As long a Partner B is made aware of WHY things are happening and still chooses to do nothing to change it, he doesn't deserve to be in a relationship with either of you. I also don't understand why in the world Partner A would even plan on staying in a relationship that he didn't feel happy in.
On that note, have you communicated to Partner B that you have put a time limit on your relationship? I have no idea how long this has been going on, but if he doesn't know that you are threatening to leave, what good does it do? Do you expect him to magically change just because you've told him you're unhappy without telling him how incredibly unhappy you are? Ignore all of that if you have informed him.
I could also see a vee being a possibility, if Partner B was willing to work on his insecurities. If you and he just aren't compatible, you aren't compatible. It sounds like he has a lot of fear and that may be where all of this is coming from. Have the three of you thought about seeking some sort of therapy or counseling to help your communication? Also, the love languages book/assessment
is really helpful for people who have different modes of showing affection.