She might be just having issues with trust, instead of actively wanting to hamper your sexual enjoyment of each other. She doesn't know you - how can she trust you in that when you say you are only having sex with him and your husband? Not only does she have to trust her husband and you, but also YOUR husband, with her health and safety. She will have to trust that your husband will never have an affair that he chooses not to tell about to you.
I had a very similar problem which did not end very satisfactorily. Female condoms didn't affect my sensation as such, but they didn't stay on very well
. Are you sure all of this is just condom anxiety and not general performance pressure, as you said he isn't really responding well to manual stimulation either?