Wow, what an amazing story. I am sorry to hear about your ex and what a terrible thing she did to you, but I think it is such a wonderful thing that your new friend was willing to help you heal and that you all took it very slowly.
I am wondering about her husband's fears that you will develop a romance. Is that something he told you directly, or did you hear it from her? Although I am always at odds with this mindset of men letting their wives be physical with someone as long as they have no emotional involvement, which is really closer to swinging or an open marriage than polyamory (which is all about the love!), it seems like you are saying that you still view it as a friendship that has a physical/sexual component.
I don't see why you cannot simply tell him, "I just want to assure you that I view ___ as simply a friend who I am physical with, and have no intention to pursue anything more than that." Maybe others with experience in this kind of relationship can give you better advice on how to say it, but I am always one for direct communication to set the record straight.
Please keep us informed on how it goes, and welcome to the forum.