Yeah, but that is actually a correlation that does match fact. A woman walking alone is far more likely to be assaulted by a man than by a woman. (Hell, I've been mugged twice, both times by a white man) The statistics actually back that one up. In a situation late at night, by yourself...sure, that's reasonable. Refusing to interact with all men because men are more likely to attack you may not be so reasonable.
That sort of thing might marginalize men if police started following or arresting the majority of men who are walking home late at night because they are men.