Hmm, so you're in a triad and being treated like a dirty little secret by the couple both in public and in front of their children. A little more info is needed, I think, in order to give more precise advice. How do they introduce you to friends and explain your presence to their kids? Do you live with them? Do you have any other relationships, or do they not "allow" you to be with anyone else? Are there any other ways in which you feel sort of set aside and not able to be yourself with regard to them?
Know that you do not have to accept and put up with crap that makes you unhappy.