I'm 100% with Natja on this. This is not the beginning of a poly relationship, this is a potential disaster with you losing all your power.
I understand you like him a lot and are prepared to engage with him and his wife, but you can't do it solely on their terms.
You say you don't have other important relationships at this stage, which suggests you have little experience of practical poly -- this isn't a good place to start! Remember they have each other, he currently has a wife and a best friend and potential lover, she wants to regulate your and his relationship, and you are thinking of pinning your future on something you have no control over.
Is that wise?