I'm sorry you're in this mess. It is obvious that the hubby and wife are not on the same page in regards to you. They are going to have to have some long, honest conversations about whether this polyamous situation is right for them. It seems as if the hubby is not really on board with this idea right now, and that's why he gets pretty jealous if you guys spends too much time (for him) with each other. It's actually pretty irresponsible of both of them to let you into this relationship before they actually agree on what they want, because you're definitely stuck in the crossfire.
Right now the hubby's attitude seems to be one of a monogamous husband who is OK with you hanging around as long as it involves a FMF threesome. And you're not wanting that. You'd prefer a V relationship where you and the hubby are not romantically involved. But the hubby is not OK with that.
It's probably wise if you step away from this situation and let them work on their expectations about such a relationship. But please don't do something like try to fool around with the wife behind his back. That could turn out really bad for everyone, as well as being unethical.