I heard "I can't do that" all the time. It doesn't bother me, nor make me feel distanced. They just couldn't do it. There was a time when I wouldn't have thought I could either. When I said I was bi a very good female friend said "Ewwww. Vagina is gross!" This and the look on her face made me laugh until my sides hurt because I knew that, while SHE felt that way about vagina, she didn't feel ANY different for me. It's like me saying I like eating cow brain and someone saying "Eww. I wouldn't". It doesn't mean we can't be just as close friends.
But I've had others tell me "you're disgusting" or "you must have a bad marriage then" or "your husband is just using you" or "you don't really love eachother then". That's where it gets insulting and people get cut off.
I'd rather be hated for who I am than loved for who I am not.