Not gonna lie, not a fan of his behavior towards you. She's obviously got an issue and is excluding you and he's allowing it. While this lifestyle does take work, it takes work on EVERYONE'S part. Including the other woman.
Talk to him again about how you're feeling, and be as CLEAR as possible that you find his behavior to be unacceptable for you. He needs to be able to split his time and attention between the two of you if it's going to work successfully. This is not a "play by my rules or you're out" kinda convo, but everyone has to be on the same page. If you're not, then get out and find someone that can manage this lifestyle. The use of the word manage is intentional.
Good luck and keep us posted.